If you have been making non-deductible contributions to a traditional IRA, you should have been filing Form 8606 each year that you made these contributions as part of your federal tax return. This is the way you and the IRS keep track of your basis (what part of total IRA that taxes have already been paid on).
When you take distributions from your traditional IRA, you use the Form 8606 to calculate what portion of the distribution is taxable and what portion has already been taxed. Essentially, the portion that is not taxed is your basis in all traditional IRA divided by the value of all your traditional IRAs as of the end of the year.
You have been filing form 8606 each year you made a nondeductible contribution, haven't you?