I think it's just human nature to hate your mortgage. Every once in awhile, I get the urge to pay it down more quickly, just for the peace of mind. I hate thinking about it, every time it comes around. All ~$2460 of it.
I keep thinking about how nice it would be to get rid of that payment, as it's a pretty big chunk. I think the human brain is just wired to look at small numbers differently from big ones, though. On one hand, it feels like a big number, but then when I compare it to my total net worth, suddenly it seems inconsequential.
Then, there's the fact that that number will never, 100%, go away completely, because it includes taxes and insurance. If I was mortgage free though, I'd pay the taxes and insurance once per year, and then forget about it. But, would still average out to around $520/mo, so the principal/interest part of the mortgage is around $1940.
Just this morning, I was thinking about if I hit a certain threshold, maybe I'd pay down 1/4 of the mortgage, just to get a jump start. But, I'd take an income tax hit if I did that. And that money wouldn't be working for me anymore, earning interest, dividends, appreciation, etc. And then, even if my mortgage was only 3/4 the original size, the monthly payment is going to be the same, unless I refinance.
Then, there's the effects of inflation. While I don't like that $1940/mo principal/interest payment, it will stay the same for the next 29 1/2 years (I bought the house about 2 1/2 years ago, but refinanced last year). Nobody can predict what inflation will do in the future, but if the past is any indication, it took roughly 30 years for prices to double. And around 20 years for prices to rise 50%. So if we take the same route, in 20 years, it should be somewhat like having a ~$1300/mo mortgage in today's dollars. And by the time I make the last payment, it should hurt about as much as a ~$960/mo payment would, now.
So, in my case at least, I think the best thing to do is nothing at all. The peace of mind of a paid off house would be nice, but I've had mortgages and/or HELOCs in my life since 1994 now. I was mortgage-free for about 7 months in 2018. I had paid off my old house at the beginning of the year, and bought a new place in September. I guess the biggest factor in paying off a mortgage, is how important is that free-and-clear peace of mind worth to you?