A couple things come to mind for your situation. First off I would say that you may be over sanding with the finer grits causing burnishing marks. Also, if you are working up through the grits keep a few things in mind, first dont start with 80... if you dont need to.. Meaning if the tool marks are barely there I wouldnt start out with anything heavier than 120. Any over aggresive grits will and often do, act like a shapping tool and will remove wood very quickly. The purpose of sand paper is to remove the sanding marks of the previous grits. If you are cutting your own sanding pads, cut them oversized and scallop the edge in a wave pattern. This will help around edges and keep the sanding pad from causing groves.

I maybe unique, but I sand very often at high speeds. Full throttle on the right angle drill/sanding pad set up and usually at the same turning speed. I am very used to this technique and only make one pass, back and forth before pulling the sander off the wood to stop heat buildup. I then blow off the piece completely with the air hose between every grit and I sand each grit using both forward and reverse on the lathe.

If your sanding at low lathe speeds, I would definitely be sanding with the drill at full throttle.
A regular shaped bowl is completely sanded and ready for finish in 10 mintues or so, doing this.

Now I understand that safety and heat issues are VERY GOOD reasons to slow the lathe down, but unless i am sanding irregular, NE or warped wood I go full speed ahead!