Let's say you bought $10,000 worth over time. Before the sale, say it was worth $12,500. If you sold $2,500, you can't skim the gains off the top and leave the basis as $10,000, if that's what you are thinking. Instead, if they used average cost, you liquidated 1/5 of your holdings, so they would reduce the basis by 1/5 of the original basis, or $2000. So, the sale of $2500 had a basis of $2000 and a gain of $500. Your remaining $10,000 has a basis of $8000.
And if some of the shares were purchased in the last 12 months, that fraction would be a short term capital gains (someone correct me if that's wrong) which you don't want because it'll be at your 15% rate, not 0%.
Does that help?