Paying off Mortgage with Taxable IRA

[Sunset]

Where do you park your cars when you go to the property by boat? I'm guessing on a town road or parking lot or a piece of land that you own that is accessible by car.... so your rationalization is BS.

Also, if you don't pay your income taxes they wil throw you in jail too... if you don't keep paying you lose your freedom... so I guess the way you think that there is no freedom.

Not sure why this bothers you.

I park on private land and pay the owner for the parking. I bet the owner pays property tax to pay for the road to his land. Nobody pays tax for driving on the road.

The oddity of property taxes has nothing to do with income tax, so bringing that up is just adding another topic.

 

[donheff]

I would recommend you ask your FA to set up a new/special account within your IRA. Put the entire mortgage amount in that account. Then have the mortgage bank, or your FA, set up the monthly payments...for however many years that 3.5% is supposed to be.
From a mindset standpoint, you have paid it off. You do not have to write a check each month...but at the end of your mortgage term, you will probably have a large sum of money to spend, give to charity, or pass on to your heirs.
You should be able to sleep well, and have a smile on your face each morning.

A DIYer could do it with a virtual account they track on a spreadsheet or, if really wanting to lock in the payoff, rollover the amount needed to a separate IRA and setup payments there. They would still need to keep the amount in a stable fund, treasuries, or something to guarantee the payoff but the tax savings would be worth the hassle.

 

[pb4uski]

Not sure why this bothers you.

I park on private land and pay the owner for the parking. I bet the owner pays property tax to pay for the road to his land. Nobody pays tax for driving on the road.

The oddity of property taxes has nothing to do with income tax, so bringing that up is just adding another topic.

The real oddity is the way you view property taxes. It bothers me because claiming that you don't really own land (or property) because you have to pay property taxes is such an ignorant way of looking at it that is foolish to perpetuate.

In our town, and I suspect yours, the cost of maintaining the public roads that you use to get to where you park to get to your island is funded by property taxes... all property owners share in the cost of maintaining the roads, contributing towards that cost based on the relative value of their properties since the tax rate is the budget (which includes the cost of maintaining the roads) divided by the grand list.

I have a very similar situation at my summer home. We are in a remote corner of the town and pay a boatload in property taxes and services are negligible... the only real services the town provides is to grade the road a few times a year and plow them in the winter.

Actually, many states have annual taxes on cars based on the cars' value... similar to a property tax... while they won't take your car if you don't pay it you just can't drive on public roads.

And technically they don't take your property away if you don't pay your property taxes... they sell it at public auction, reduce the proceeds from the sale for the property taxes in arrears and then you would get a check for any excess of the sale proceeds over the property taxes in arrears... then same as if you didn't make your mortgage payments and were foreclosed on and the sales proceeds exceeded the amount of your mortgage.

 

[ut2sua]

I would recommend you ask your FA to set up a new/special account within your IRA. Put the entire mortgage amount in that account. Then have the mortgage bank, or your FA, set up the monthly payments...for however many years that 3.5% is supposed to be.
From a mindset standpoint, you have paid it off. You do not have to write a check each month...but at the end of your mortgage term, you will probably have a large sum of money to spend, give to charity, or pass on to your heirs.
You should be able to sleep well, and have a smile on your face each morning.

This idea is pretty cool. However, I wonder if stock is a part of this special account (?) If so, if the market drop 50%, how does the OP sleep well then? Also, there would be on-going tax calculation to be worked out every year (I understand that the total tax to be paid is less than the "lump sum" withdrawal OP has planned). If stock is not part of the special account, then the remaining $ may not be that large.

 

[levindb]

Let me address the tax situation first:

1. If the OP takes a large distribution to pay off the mortgage, then there is a significant likelihood that the amount withdrawn can push the OP into higher tax bracket(s) since the balance is so large.

2. The smaller annual withdrawals over 10, 20 or even 30 years will has less of on impact on taxes. Taxes need to be paid no matter what.

3. One reason to accelerate the payoff of the mortgage may relate to the changes n the tax tables planned for 2026.

On the risk of a market collapse:

1. Yes, that is a possibility. The mortgage is not the only thing that will have to be dealt with...so will living expenses. Having a larger total portfolio because the mortgage funds were NOT removed early, should make life better for the OP because he would have more flexibility to decide how he pays for food.

2. If the overall nestegg value dropped by 50% there will probably be sleepless nights ahead, there is no doubt about it. Having paid off the mortgage may let him sleep better, but now he needs to figure out how to buy food...which could keep him awake all night.

Overall, one has to manage risk and manage taxes...if, however, the emotional side of things controls ones life, then maybe they should not retire early; maybe they should not invest in stocks; and, maybe they should buy an annuity

 

[Can_I_Retire_Now]

Let me address the tax situation first:

1. If the OP takes a large distribution to pay off the mortgage, then there is a significant likelihood that the amount withdrawn can push the OP into higher tax bracket(s) since the balance is so large.

2. The smaller annual withdrawals over 10, 20 or even 30 years will has less of on impact on taxes. Taxes need to be paid no matter what.

3. One reason to accelerate the payoff of the mortgage may relate to the changes n the tax tables planned for 2026.

On the risk of a market collapse:

1. Yes, that is a possibility. The mortgage is not the only thing that will have to be dealt with...so will living expenses. Having a larger total portfolio because the mortgage funds were NOT removed early, should make life better for the OP because he would have more flexibility to decide how he pays for food.

2. If the overall nestegg value dropped by 50% there will probably be sleepless nights ahead, there is no doubt about it. Having paid off the mortgage may let him sleep better, but now he needs to figure out how to buy food...which could keep him awake all night.

Overall, one has to manage risk and manage taxes...if, however, the emotional side of things controls ones life, then maybe they should not retire early; maybe they should not invest in stocks; and, maybe they should buy an annuity

The decision at this point is to double up on the mortgage payments and allow what we have to continue to grow over time. If at some point we feel/believe that things will go south economically, we will just pay it off and suffer the tax consequences. As far as how to pay for food, this is one of the things why we bought a farm - is to get to a place of self sustainment so we don't have to rely on the system to provide us with the basic necessities.

 

[JP.mpls]

Can I Retire Now,
I emotionally feel the same way you do. Regardless of financial logic, it is very satisfying to have no debt.



When my wife and I were still paying for our primary home, we started paying extra principle each month, and we did end up paying off a 30 year loan in about half the time. When it was paid off, we started maxing out our Roth accounts with the extra monthly money.



I believe if there is an advantageous time to pay down a mortgage, it is in the first third of the loan term, when each payment is primarily going heavily towards interest.



I'm in the same position as you. I recently purchased a second home, and took out a 30 year mortgage. I'm not paying it off immediately, but I did end up paying off a big chunk of it with money that I had in a Roth account. I still have a decent amount in my Roth accounts for liquidity.



It reduced the loan from a 30 year loan to a 11 year loan. My plan is pay the remaining amount monthly for the next 11 years. A compromised approach?



Take care, JP

 

[ut2sua]

Let me address the tax situation first:

1. If the OP takes a large distribution to pay off the mortgage, then there is a significant likelihood that the amount withdrawn can push the OP into higher tax bracket(s) since the balance is so large.

2. The smaller annual withdrawals over 10, 20 or even 30 years will has less of on impact on taxes. Taxes need to be paid no matter what.

3. One reason to accelerate the payoff of the mortgage may relate to the changes n the tax tables planned for 2026.

On the risk of a market collapse:

1. Yes, that is a possibility. The mortgage is not the only thing that will have to be dealt with...so will living expenses. Having a larger total portfolio because the mortgage funds were NOT removed early, should make life better for the OP because he would have more flexibility to decide how he pays for food.

2. If the overall nestegg value dropped by 50% there will probably be sleepless nights ahead, there is no doubt about it. Having paid off the mortgage may let him sleep better, but now he needs to figure out how to buy food...which could keep him awake all night.

Overall, one has to manage risk and manage taxes...if, however, the emotional side of things controls ones life, then maybe they should not retire early; maybe they should not invest in stocks; and, maybe they should buy an annuity

All your points are fair. I take the last paragraph as your personal view point which is fine. Personally, I slightly disagree with that as I think the emotional side of things is as important for someone's well being (particularly in retirement). I just want to add the above to color up the discussion (to be fair, you didn't ask for my opinion ). Thanks for your response.

 

[SonofCohoes]

The decision at this point is to double up on the mortgage payments and allow what we have to continue to grow over time. If at some point we feel/believe that things will go south economically, we will just pay it off and suffer the tax consequences. As far as how to pay for food, this is one of the things why we bought a farm - is to get to a place of self sustainment so we don't have to rely on the system to provide us with the basic necessities.

If I had enough to pay it off and take a 30% tax hit I’d think things through a bit further.

Have your FA set up a withdrawal plan do the mortgage payment and any associated fed and state income taxes on the withdrawals automatically paid. You don’t see the bills monthly and in reality all you should be thinking about is replenishment of the interest paid on the mortgage as the principal, as I see it moves from one asset to another.

Just my .02.

 

[68bucks]

I have been on that pay off or not fence for some time. Of course my FA is against it, I think mainly because he will lose that money under management. I know the math doesn't make sense to pay it off. We bought 5 years ago and I got a 2.875% 5/1 Arm. I told him I wanted to pay it off in 5 years. Well I just closed a refi on Monday to a 15 yr fixed at 2.75%. I'd still like to pay it off but see the benefits of not, including the tax issues you see. For now I'll keep the mortgage but told my FA he needs to make 2.75% plus his cost to break even. I will still pay it off in the next 5 years or so but as some mentioned I'll do it in bits to avoid the big tax hit. We have the ability to pay it off whenever we want so I can live with that until it makes more sense too. If you're in the same situation I'd just keep the mortgage and see how it goes for you. Just an FYI I retired in October 2018.

 

[rayvt]

Maybe the right answer for us is to double up on the payments (additional going to principal only) for the immediate term. This allows us to stay liquid while at the same time keeping us in a lower tax bracket.

No, don't do that. Legally, there are only two states for a mortgage: 1) You have a mortgage. B) You do not have a mortgage.

There is no state C: "A mortgage with a lower balance than originally scheduled." That is known as: Having a mortgage.

If you want to do that, take the money out of your IRA and put it in a savings account, call it your Mortgage Freedom Account. Leave it there until you can completely pay off the mortgage.

As others have already said, financially you are best off keeping the mortgage. If you decide to do the financially sub-optimal thing and pay it off, at least do it smart and not dumb.

Your FA is right. Set up an auto-pay for the monthly mortgage payments and let it run. You only need to touch it once a year, when the new escrow amount is updated.

 

[pb4uski]

.... Set up an auto-pay for the monthly mortgage payments and let it run. You only need to touch it once a year, when the new escrow amount is updated.

Or better yet, request to be removed from escrow (we haven't had escrow12 for insurance or property taxes since 1986)... then you never have to touch it... set it for an amount of payments equal to your remaining number of payments and forget about it... that's what we did from 2012 until we paid it off at the end of 2019.

Actually, we did have to touch it a couple times when the servicer changed and we had to change the address that the monthly auto-pay was sent to.

 

[rayvt]

We have the ability to pay it off whenever we want... Just an FYI I retired in October 2018.

If you have the ability to pay off the mortgage whenever you want, do you really have a mortgage? Literally, of course you do. But practically its more like you have $XXXX in liquid(ish) investments assets of which $XX is earmarked for the balance on your house.

You never have to lose sleep worrying about "How can I pay the mortgage if I lose my job?" when the answer is always "Simple, just call up the stockbroker and have them cut a check."

FWIW, the easiest mortgage refi I got was the time when the loan processor noted to the underwriter, "Note that his 401K balance is larger than the requested loan amount."

 

[RunningBum]

I have been on that pay off or not fence for some time. Of course my FA is against it, I think mainly because he will lose that money under management. I know the math doesn't make sense to pay it off. We bought 5 years ago and I got a 2.875% 5/1 Arm. I told him I wanted to pay it off in 5 years. Well I just closed a refi on Monday to a 15 yr fixed at 2.75%. I'd still like to pay it off but see the benefits of not, including the tax issues you see. For now I'll keep the mortgage but told my FA he needs to make 2.75% plus his cost to break even. I will still pay it off in the next 5 years or so but as some mentioned I'll do it in bits to avoid the big tax hit. We have the ability to pay it off whenever we want so I can live with that until it makes more sense too. If you're in the same situation I'd just keep the mortgage and see how it goes for you. Just an FYI I retired in October 2018.

There are some real inconsistencies here. You recognize the math isn't good to pay it off, and see the benefits of keeping the mortgage, but accuse your FA of looking after their own interests rather than the other factors. If you really distrust your FA this way, you should look into getting one with fiduciary responsibility, or manage your investments yourself.

Then you say your FA better get you 2.75%+fees. Or what? If the market does drop and you fall short, you're going to sell your investments at a low? If that's your mindset, you should just sell now when things seem high to most people.

Both the pay-off and keep the mortgage decisions have merits, but your decisions should be based on better logic than this. IMO your decision should be based on which of these two arguments make better sense for your situation.

1) The market seems high, so I'd rather lock in a guaranteed 2.75% "return" by selling high and paying off my mortgage. This will not leave me short of cash for unplanned expenses, and does not give me a bit hit with taxes to put together the cash to pay off the mortgage.

or

2) Over the remaining life of the mortgage, I have a lot of confidence that I can beat the 2.75% mortgage rate. I don't want to be house-rich and cash-poor. It would be a big hit to withdraw enough funds from a retirement account or capital gains to sell from a taxable account. It makes more sense to spread out the mortgage over it's lifetime.

 

[kgtest]

Refinance to a 10-15 year to save on interest if possible? Then, take the extra $ you were paying to the lender in interest charges, and instead pay down your mortgage notes principle.

Then, make an extra payment or 2 from IRA each year ONLY IF you are still not sleeping.

Only do this if the break-even calculator works in your favor after adding in refi fees.

 

[HI Bill]

2) Over the remaining life of the mortgage, I have a lot of confidence that I can beat the 2.75% mortgage rate. I don't want to be house-rich and cash-poor. It would be a big hit to withdraw enough funds from a retirement account or capital gains to sell from a taxable account. It makes more sense to spread out the mortgage over it's lifetime.

Regarding the house rich/cash poor idea: I'm in the middle of buying a home. We are currently financing half. I've thought of paying it off over a few years, but that would mean I have fewer invested assets. Paying it off means higher taxes, but no more earning (or losses) on the payoff amount. Depending on one's cash flow sources and age, and remaining assets, and WR, paying off a mortgage may or may not make sense. For us, (ignoring taxes and potential earnings/losses), either way, we would be about cash-flow neutral.

I just went through this with my 84 year-old father. The hit in taxes and the hit on his portfolio balance to pay off his mortgage would mean that his portfolio would drop by ~75%. But he would have no more mortgage payment, and his month expenses would go down by ~$500 per month. IF he believed that the markets could tank again, then biting the bullet over a two-year period and paying it off makes the most sense. He didn't want the tax bite, so decided to pay off his car instead, freeing up ~$300/month, and not costing him much in taxes.

 

[Out-to-Lunch]

Slight threadjack: Our mortgage is close to being paid off. This thread made me curious, and I just looked to find that, as of the Aug. payment, there is $0.36 more money in my escrow account than principal left! I was planning to wait until January to pay this off, but maybe I should just pay this off now, and learn to pay my own property taxes . There is only ~$50 bucks of remaining interest hanging in the balance here, but why not?

(Due to the low stakes, I am not actually requesting an analysis. Just sharing some happy news that the thread evoked.)

 

[Retired Expat]

I recently paid off my mortgage with short term interest only loan being offered by my brokerage firm, secured not by the property but by investments. 3 years at 1.5% no fees or points. Ask your broker if you could secure a low rate loan with the IRA. Take the monthly savings and invest them and take money each year out of the IRA to be able to pay off the loan without the one time tax hit. It was always my intention to pay it off much sooner than the mortgage term as I too hate the idea of a mortgage, but if investments were making 8% ROI and a mortgage cost 4% It is hard to justify using cash that could be invested.
,

 

[68bucks]

There are some real inconsistencies here. You recognize the math isn't good to pay it off, and see the benefits of keeping the mortgage, but accuse your FA of looking after their own interests rather than the other factors. If you really distrust your FA this way, you should look into getting one with fiduciary responsibility, or manage your investments yourself.

Then you say your FA better get you 2.75%+fees. Or what? If the market does drop and you fall short, you're going to sell your investments at a low? If that's your mindset, you should just sell now when things seem high to most people.

Both the pay-off and keep the mortgage decisions have merits, but your decisions should be based on better logic than this. IMO your decision should be based on which of these two arguments make better sense for your situation.

1) The market seems high, so I'd rather lock in a guaranteed 2.75% "return" by selling high and paying off my mortgage. This will not leave me short of cash for unplanned expenses, and does not give me a bit hit with taxes to put together the cash to pay off the mortgage.

or

2) Over the remaining life of the mortgage, I have a lot of confidence that I can beat the 2.75% mortgage rate. I don't want to be house-rich and cash-poor. It would be a big hit to withdraw enough funds from a retirement account or capital gains to sell from a taxable account. It makes more sense to spread out the mortgage over it's lifetime.

I trust my FA, who is a fiduciary, is looking out for our best interest. The smart financial thing is keep the mortgage but there is the extra plus to him managing the money. So that's not an issue. We won't be rushing off to sell when the market drops, that would be dumb. Probably the biggest reason we don't just pull the money and pay it off is the big tax hit. Push come to shove we could do that though so that is how I rationalize keeping a mortgage.

 

[CRLLS]

After several years of hemming and hawing, we just paid off our mortgage yesterday with 20 years left. We used money from our Roth. I know the math is pro - mortgage. It reduces our monthly nut, which eliminates us having to pay taxes compared to withdrawal from a tIRA. We have little investments beyond tIRA and Roth's. Our WR to cover all expenses this year was running around ~ 1%. Now it can be lower if necessary. This works for us.

Different strokes for different folks. I don't believe there is much difference between paying off a mortgage from tIRA or a Roth for those who are living off retirement investments and have a low WR. There are a lot of moving pieces to consider. All of which are unknown and subject to change in the future. Take a best shot at the moment and move forward.