Should we pay the mortgage off

[W2R]

Thank you all for your thoughts and suggestions. Very helpful indeed. I talked it over with DW. We agreed to stop plowing money into the market for a while and instead apply all excess cash flows to extinguish our mortgage.

So glad she is seeing things your way! That's terrific.

 

[Finance Dave]

DH & I have not paid off our mortgage and don’t intend to pay it off early. We have a 30 year fixed loan at 3.375%. The opportunity cost to pay it off is too high as we’ve been earning much more than that on our investments. And the monthly payment is a small part of our monthly outflow.

But using that logic, why then wouldn't you go out and borrow as much as you can against the mortgage (via a HELOC or 2nd mortgage) and invest that in the market?

IMO this ignores the potential for a market downturn and introduces risk.

 

[harley]

Thank you all for your thoughts and suggestions. Very helpful indeed. I talked it over with DW. We agreed to stop plowing money into the market for a while and instead apply all excess cash flows to extinguish our mortgage.

My only issue is (and your wife seems to agree) that by moving the money into equity in your home you lose financial flexibility. Since the after tax investments are earning pretty close to the after tax interest payments, it's nearly a wash. So to me keeping that much money out of being locked up as home equity would be the deciding factor, and worth whatever small amount you might be paying. You'll still have it paid off in 10 years when you are considering downsizing.

 

[USGrant1962]

The deciding factor for me was the financial flexibility generated by paying it off. Not having a mandatory draw on your portfolio of $10(s) of thousands per year provides huge flexibility in managing AGI/MAGI/ACA/American Opportunity Tax Credit/0% LTCG/etc and your tax bracket.

But this only works well if paying off your mortgage is not a large fraction of your taxable assets.

 

[clobber]

We will be entitled to 11K of additional deduction if we itemized vs standard.

Why/how is that? I have not followed it to closely (yet), but how are your itemized deductions affected here?

 

[Badger]

I like the idea of owning my own home free and clear. When I was a little through half the 30 year mortgage and could no longer benefit from using the interest for itemizing and had to start taking standard deduction I started making an extra principle payment each month equal to the regular monthly payments. Got it paid off in about 5-6 years and saved a little money in the process. It might not have been the best move. I would not know being a bear of little brain. But I also continued to take the same amount of money from both payments and added them to my retirement accounts. Now we are sitting pretty with no house payments. And since we get Tricare for Life and Medicare we have no medical bills either. It helped to have a plan years ago.

 

[HadEnuff]

I have been faced with this decision twice, in the last 15 years. However, both times it was a considerable smaller number. The first time the interest rate was variable, hovering around 5%. I paid it off because I wanted to be debt free. Then, about 4 years later I decided to plow about 125K into transforming my old "fish camp" style (read "about to fall into the water) lake cottage into a place we'd eventually call home, and it was such a PITA to get a mortgage.
Just by sheer dumb luck we decided to this when we could get a 15 year fixed at 2.75%., and I was still earning a nice income, so it wasn't all that much interest, after the tax break. By the time I retired the balance was around 100K, and the market was doing nicely, but mostly, I didn't want to take 100K of my after-tax money and lose the flexibility it allowed me.

However, 650K at 3.625%, with a reduction in the tax break, I think I'd like to get at least a large portion of that cleaned up. While you are doing that, you may get a feel for what interest rates are doing. At some point along the way the landscape may change, and 3.6% may be a bargain.

 

[joeea]

Thank you all for your thoughts and suggestions. Very helpful indeed. I talked it over with DW. We agreed to stop plowing money into the market for a while and instead apply all excess cash flows to extinguish our mortgage.

So instead of paying 3.625% interest on the mortgage you are forgoing $650K worth of investments at your 60/40 asset allocation?

We all do what we have to do in order to sleep at night. Good that you both agree to a solution.

 

[Andre1969]

The decision of whether to pay off a mortgage or not is a hard call, and I have a feeling there really is no right or wrong answer.

I went through a somewhat similar thing back over the summer of 2016. In my case, I owed about $140K, on a 10-year fixed at 4.99%. I wanted to refinance, but the mortgage company tried to make me jump through a bunch of hoops and I refused (mainly making some expensive repairs that I thought were not needed, and would exceed the benefit of the refinance).

I was a bit afraid to pull $140K out of the market all at once to pay it off, though. The monthly payment was $1847, but what I tried to do, was every month the value of my investible assets hit a new $10K threshold, I'd make a $10K payment on the mortgage. If it ended up being an up month, but not a new $10K threshold, I'd pay $3K. And in a down month, I'd pay $2K.

Well, the market took off in late 2016, and then I received a rather sizable inheritance in 2017, so I ended up making my final mortgage payment last week.

I think if I had a large mortgage, at a fairly low rate, and a lot of the payment was interest that I could write off, I might keep the mortgage. But in my case, the rate was a bit high (4.99%) and being a 10-year, it was mostly principal, so not much to write off. Also, being such a short term, the payment was high, and it was hurting my ability to get other financing. Lenders were just looking at my income and expenses, and not taking my net worth into account.

 

[ShokWaveRider]

We are in the Paid Off Camp. Paid ours off in the mid 90's. It was worth it as the increase in home price was tax free (Below the Threshhold). This allowed us to have more After Tax money for retirement.

 

[Cobra9777]

We paid off the mortgage when I retired 5 years ago. We were only 9 years into a 30 year mortgage. But we had made large additional principal payments for several years when I received my annual bonus. So the payoff was fairly small as a percentage of NW. But the payments were still quite large as a percentage of annual spend. That was not something I wanted to deal with early in retirement at 52.

We considered refinancing to a very low rate (around 3.25% at the time) with another 30 year loan. I ran the numbers in FIRECalc and it looked pretty favorable, although the benefit was rather negligible in the grand scheme of things, and certainly not without *some* risk. Plus we were looking at several $K of closing costs that seemed like a complete waste at that point. So paying it off was just the simpler solution for us.

 

[Sunset]

Thank you all for your thoughts and suggestions. Very helpful indeed. I talked it over with DW. We agreed to stop plowing money into the market for a while and instead apply all excess cash flows to extinguish our mortgage.

Glad to see you both agree on a financial plan.

I personally would do this, I've not had a mortgage for a long time.
Based on history, if the market tanks 30% - 50% , I would be tempted to stop the aggressive paying off of mortgage, buy broad stock etf's for a couple of years, and go back to paying off the mortgage if or when the market recovered.

Yes, at times I have been a DMT (dirty market timer).

 

[Happyras]

I enjoyed this discussion, so here is my two cents;
If the mortgage rate is >4%, and itemizing the interest is not an option due to the new standard deduction, I would definitely pay off the mortgage, most but not all, from the bond side of my investments.

In 2016, I restructured our debt to avoid drawing pre-tax investments. Since HELOC is no longer deductible, I have essentially paid off that 4% debt this month, but I refi'd our Home at 2.625% on a 10/1 at that time in Dec 2016. We have low interest, low payments and the 10/1 adjusts after I am forced to take RMD's which I will use to pay it off at that time. Since we can now barely itemize, I am walking the same line but recognize the flexibility in keeping the debt as long as our pre-tax bond allocation keeps up with a higher return (> marginal tax rate difference). For us that would be a yield of 3.25%. We have to hope for gradual interest rate change, or our duration risk is greater than our equity risk.

We have no other debt on rental or commercial properties, primarily due to the higher rates to borrow. That was our focus pre-FIRE.

 

[Cpadave]

Yes, “peace of mind” is definitely one of the factors in our decision to pay off the mortgage. What bugs me more is the negative interest spread. On the other hand though, our fixed income portfolio is a bunch of individual bonds that we bought over time, laddered in maturity, and initially intended to hold to maturity. To sell $650k of bonds would not only upset the AA balance, it would also generate friction cost in bid/ask spread, not to mention a little bit of capital loss due to the recent rate uptick. so we decided to meet somewhere in the middle. Instead of liquidiating some of the bonds, we would just use future distributions from our “externally managed” portfolio (outside of our 60/40 framework) to pay down the mortgage. Obviously this carries a lot more uncertainty in terms of time frame, but it is the least amount of work, and gives us more time to see how the equity market and rates evolve from here.

 

[Sandy & Shirley]

If you are a middle to upper middle income American, then paying off your mortgage could definitely be extremely advantageous to you. If your working income levels are in the six digit category, not so sure.

Since we are middle income, we pay a lot of attention to the 49.95% and 40.7% marginal tax rates that we all face as we enter the 22% Federal Bracket during retirement.

Here is an example that I created for one of our single friends:

She is a widow with a mortgage payment of about $1,500 a month. $300 of that goes for taxes and insurance which she will have to continue to pay even without the mortgage and $1,000 a month goes toward principal and interest.

She is currently getting survivor benefits and will switch to her own at age 70. Her final benefit level should be about $36,000 and she said she could live very comfortably on $5,000 a month, $6,000 a month if she still has her mortgage.

The chart shows a red tick mark for the taxable income required without the mortgage and a blue one if she still has the mortgage.

Her current income levels on survivor benefits puts her well over “The Hump” marginal taxes so she is back in the standard 22% Tax Bracket.

Her choices are to pay off her mortgage with IRA / 401K withdrawals at 22% and maybe 24% or keep the mortgage and pay huge Hump taxes after she is 70.

Looking at the income and tax levels above the graph, keeping the mortgage would require her to withdraw an extra $18,777 each year to get the required $12,000 after tax needed for her mortgage PI payment. The tax increase for those extra IRA withdrawals would be $6,777 which is an overall marginal rate of 36.1%.

Bottom line, is it better to pay the IRS 22% or 24% now, or give them 36.1% later.

She is near the end of her mortgage, so the possible interest deductions would not mean a lot.

These percentages are all different for everyone based on your Social Security benefit level and the standard of living that you are used to.

 

[brokrken]

Sandy, maybe I missed something, but where is the marginal tax rate 50% for someone making 70k?

 

[Lawrencewendall]

I'm not seeing it either but I may just be clueless.

https://www.mortgagecalculator.org/calcs/marginal-tax-rate-calculator.php

 

[LXEX55]

I am personally going to pay mine off, although it is much smaller than yours. There are arguments on both sides, but how about the satisfaction of knowing, you are no longer in debt?

 

[Sandy & Shirley]

Sandy, maybe I missed something, but where is the marginal tax rate 50% for someone making 70k?

The 1983 amendment to the social security act changed the “tax free” nature of your Social Security benefits to “tax deferred”. The “basis” for this taxation is one half of your benefits plus your other taxable income. This is represented on the graph by the dotted green line. This taxation starts for an individual at $25,000 and $32,000 for a married couple.

Half of $36,000 is $18,000 plus $7,000 equals $25,000. So the dotted green line starts at $36,000 plus $7,000: $43,000 on the graph. It starts at 5%, representing the 50% taxability level of your benefits. The 1993 Budget Bill added a second 85% taxability level starting at $34,000 for a single person and $44,000 for a married couple.

Note how the solid blue line starts at 15%. If you earn / withdraw an additional $100 when your tax bracket, the dotted red line, is 10%, that $100 makes $50 of your deferred social security benefit taxable so you are paying 10% of $150, not just the $100 that your income increased and $15 is 15% of the $100 that you withdrew. The next marginal tax level is 18.5% when the taxability level is 85% and your tax bracket is 10%. The next marginal tax level is $22.20 which is 12% of $100 plus $85.

Let’s ignore the 49.95% level and jump to the 40.7% level. 22% of $185 is $40.70.

Now to the more complex 49.95% marginal bracket. The 22% bracket starts at $38,700. For some reason the new tax bill starts the taxability of your deferred LTCGs at $38,600. This is why there is a small dip in the marginal tax line over that $100 income level.

Assume that your taxable income is at $35,600. Plus your $3,000 of LTCGs is a total of $38,600, nothing over that so none of your gains are taxable. Now, you withdraw an additional $100 from your IRA which makes an additional $85 of your Social Security benefits taxable. So your taxable income increases by $185. $185 at the 12% tax bracket is $22.20. But, this also pushes $185 of your tax deferred LTCGs over the $38,600 limit where they are now taxed at the special 15% bracket. 15% of $185 is $27.75.

Bottom line: When the taxation of your additional income is added to the deferred taxation of your Social Security Benefits plus the deferred taxation of your LTCGs, $22.20 plus $27.75 results in paying an additional $49.95 because you withdrew $100 and that is a 49.95% marginal tax bracket.

So, to correct your statement, the IRS is not taking 50% of your money, they are only taking 49.95%.

 

[brokrken]

Aah, got it now. Although I don't fully follow your logic (why are you adding $7K?) I see your point that when making these decisions you have to consider all of the additional taxes that will be triggered by that extra income that you choose to take.

 

[Sandy & Shirley]

Aah, got it now. Although I don't fully follow your logic (why are you adding $7K?) I see your point that when making these decisions you have to consider all of the additional taxes that will be triggered by that extra income that you choose to take.

The taxability of a single person’s Social Security Benefits start when the “basis” is over $25,000. The “basis” calculation is one half of your Social Security Benefit plus your other taxable income. Since half of your $36,000 Social Security Benefit is $18,000, you need an additional $7,000 of other taxable income to reach that $25,000 level.

So the point where your basis reached the taxability point is $18,000 plus $7,000 and the reality for your actual income is $36,000 plus $7,000. You are using half for the basis and all of it for your actual gross income.

 

[Sandy & Shirley]

Here is another graph that I posted on a different thread that might be helpful:

It shows how larger benefit levels result in higher gross income levels before you start paying taxes, you are getting more “tax deferred” income, which result in more tax savings.

The other side of this is that the more taxes you save, the more the IRS wants back, which results in larger “Tax Humps”.

You don’t need all the fancy graphs to do your planning, they are just the old cliché picture that in this case is worth thousands of dollars. What you need to know is that these humps start at either the $38,700 22% taxable income level or the $38,600 taxable LTCGs level.

 

[COcheesehead]

I am building a new "retirement" house and we plan to pay cash for it. The funds will come from the sale of our current home and short term investments we now hold. The way I look at it is I will need about 14% less money each year by not having a mortgage based on our planned retirement budget. That allows me to dial down my market risk and still have all the money I need to pay the bills and have fun.

 

[Sandy & Shirley]

I am building a new "retirement" house and we plan to pay cash for it. The funds will come from the sale of our current home and short term investments we now hold. The way I look at it is I will need about 14% less money each year by not having a mortgage based on our planned retirement budget. That allows me to dial down my market risk and still have all the money I need to pay the bills and have fun.

Not sure what “all the money I need” means, but like I said, try to do your taxes as if you are retired now and see how close you will be to the 22% Federal Bracket. If you are close to it, doing Roth Conversions now at 22% is far better than taking the money out later and paying 49.95% and 40.7%.

Note the difference in the two lines above with $24,000 SS Benefits. You are taking $7,908 less out of your IRA and replacing it with $4,992 from tax free sources which reduces your taxes by $2,901.

Paying $2,901 on a $7,908 is an overall 37% marginal tax rate; some at 49.95%, some at 40.7%, and some at 22%.

 

[COcheesehead]

Not sure what “all the money I need” means, but like I said, try to do your taxes as if you are retired now and see how close you will be to the 22% Federal Bracket. If you are close to it, doing Roth Conversions now at 22% is far better than taking the money out later and paying 49.95% and 40.7%.

Note the difference in the two lines above with $24,000 SS Benefits. You are taking $7,908 less out of your IRA and replacing it with $4,992 from tax free sources which reduces your taxes by $2,901.

Paying $2,901 on a $7,908 is an overall 37% marginal tax rate; some at 49.95%, some at 40.7%, and some at 22%.

I can't do conversions now, I am in too high of a tax bracket. I hope to show as close to 0 income as I can in my first two years of retirement and plan to convert in those years. SS income is way down the road for us.