But if early payoff requires pulling $ from tax deferred accounts, because that's where the bulk of one's assets are parked, what is a good strategy for doing so? Is it the same calculation as a Roth conversion, only you put the converted $ toward the mortgage instead of a Roth? Has anyone here done it?
Our mortgage balance is $280,000, which would take $450,000 ballpark to generate the principle and interest payments sustainably at a 4% SWR, so there is some short term appeal to paying it off, which I realize dissipates longer term as that $450,000 "endowment" grows faster than the mortgage payments.
You should seek professional guidance on this, but here is my take: I think the main thing is not to venture too far into the next tax bracket because of your withdrawals - unless you will find yourself in that bracket when RMDs begin. It can get a little complicated because RMDs "accelerate" as you age AND you don't know what future value you will have in your deferred account (performance of your AA will vary.)
Really makes no difference (except the way it's handled) with a Roth or just taking the money - same calculation. You will still owe the taxes on what you take (assuming you're not at zero tax).
Here are at least some of the complicating factors, no matter what you decide to do:
1) You do not know what future tax policy will be.
2) You do not know what performance your funds will have
3) You do not know what "gotchas" may be added or subtracted (see 1)
4) You do not know how long you will live.
1) You do know what your RMD multiplier will be (Assuming it's not changed in the tax code).
2) You do know that there will be AGI gotchas
3) You do know that you are currently "overweighted" in deferred money vs cash or other "already taxed money, e.g., Roths.
What I have been doing is attempting to get to a place where I have more control in the future. If the tax rules change or gotchas change, it's MORE likely that you will wish you had less in deferred and more in non-deferred. Of course, you don't know that for a fact. It's complicated, but I see it as playing the odds. For instance, the Feds could "double tax" your Roths. Probably less likely than more or worse AGI gotchas (they are less visible to the public.)
Again, do research or get help if you don't feel you know the answers or the details involved. Get this right and you might save quite a bit, especially in the future. Still, YMMV.