Check out form 8606 http://www.irs.gov/pub/irs-pdf/f8606.pdf
which you will need to file for your non-deductible contribution and also to calculate the tax on conversion. Although you can convert as you suggest, IRS considers all your TIRA accounts as one and you will have to treat the amount you convert as a mix of both deductible and non-deductible contributions ...the deductible part will be taxable.
Pls note that form is for 2009. Add 1 to all the year dates for 2010.
You may also want to explore this idea:
“Isolating basis” (the non-deductible contribution amount) can be
accomplished by rolling pre-tax IRA portion into employer plan or solo
401(k). You thus will in effect be ‘hiding’ that portion from inclusion in the
conversion tax calculation, and voila, the Roth conversion of the after-tax
only portion will be tax free – pure gold. But tread very carefully.
(I don't know what the tread very carefully entails).
see strategy 4 here: http://www.fairmark.com/rothira/09030801-401k-basis.htm