When you sharpen, ensure that the burr on the flat side is totally gone. I would imagine that this is being mistaken for the edge folding over and will be much more pronounced after light use if it remains. I also suspect this is common with old chisels that need some real effort to re-flatten.
Chopping cuts on a clean edge should create a fine finish. If you want to test your edge, forget about shaving your arm. Chop near the end of your cut off stock, bevel down, the notch it creates should show a smooth finish. If there are lines in the finish then your sharpening needs improvement.