Some quick basics mostly off the top of my head. Others can correct any old or incorrect info.
There are different formats for the music. You can convert your CDs to mp3 format that can be played on any brand player. I think iTunes sells songs in aac or mp4 format, which only iPods can play. The other download sites do mostly wma format with drm licensing, and iPods can't play those. So pretty much you have to decide whether to go with Apple iPods or not, and stick with that if you're buying a lot of music. I guess there are ways to convert between formats but I haven't figured it out yet. It sucks, it's like vhs vs. beta before it was clear that vhs would win out.
Some sites are starting to forgo licensing and sell mp3 format. Hopefully that will catch on and make switching easier. Of course there are all the free file sharing sites that you can get mp3 music from, but that's also not legal, and some sites are known for being virus prone. That's all I'll say about that.
Most people find that iPods are easiest to use. They are also more expensive than the other brands.
Then you want to decide between a flash memory player or hard drive.
Flash memory tops out at 8MB right now, I think. 1GB of memory gives you about 16 hours of music in mp3 format. If your goal is to put your entire collection on a player, a flash player may not be enough, though in your case unless you are going to buy a lot of new music it may be fine for you. And if you want to stay at around $80 it's about your only choice (but see my comment about a Zune later).
Hard drive players arent really good to use while exercising, because of the weight and also the bouncing can shorten the life of the hard drive. I have both kinds myself.
Pricewise, someone recently posted a Woot one-day special on a reconditioned Microsoft Zune 20GB player, I think it was $85 with shipping and tax. I got one even though I already had a Dell, as a backup/spare/whatever. So far I'm not liking the usability of moving music because it "syncs" music and I can't get it to find music that I've put on an external hard drive. I like the drag and drop method better.
Woot also recently had a 2GB Sansa player for about $25 plus shipping and tax. Also reconditioned. I picked up one to give as a gift, but haven't received it yet.
I think you'll have a tough time finding that low of prices everyday, and certainly not for new. I think $60-100 is more typical for flash players, and $200 or more for hard drive players. Amount of memory is a big factor, of course.
Both also have video capacity, which I don't care for so much (I can wait until I get home to watch movies or shows) except that it seems nice to store a few pictures, kind of like carrying photos in your wallet.
