My father-in-law is in a similar situation with a variety of before-tax and after-tax 401(k) contributions.
Unfortunately his lump-sum pension and his 401(k) were rolled over into an IRA that already held deductible and non-deductible contributions.
This year is his first RMD and it's been pretty much a mathematical nightmare. My brother-in-law the CPA is getting mightily tired of chasing down all the years of contributions & their gains.
If your IRA custodian has any sympathy for customers, they'll allow you to set up separate accounts for each flavor of contribution. Another way would be to put each type of contribution in their own separate mutual funds/ETFs/stock portfolios.
If not, you'll need to keep careful track of the cost basis of each and ratio out the investment gains. And you'll have to do the same tracking if you roll the funds over to a Roth.
I have an $1700 pre-tax TSP account that I rolled over into my conventional IRA without realizing the impact of that "decision". Now that I'm converting the six-figure IRA to a Roth I'm wondering if it's even worth tracking down that small contribution. No doubt the IRS will tell me if I care...
The book written on E-R.org, "The Military Guide to Financial Independence and Retirement", on sale now! For more info see "About Me" in my profile.
I don't spend much time here anymore, so please send me a PM. Thanks.