Teacher Terry had it right, saying: "We actually paid off ours 5 years ago but recently were sorry because so much $ tied up in the home"
supernova72, if you paid the house off from your 401K your asset allocation would be $690K investments and $565K house. That's 45% of your net assets in one piece of illiquid, non-income-producing real-estate and 55% in investments.
That strikes me as a pretty bad asset allocation.
If your mortgage is indeed a 15 year, that was your first mistake. A 15 yr mortgage commits you to a substantially larger monthly payment than a 30 yr. You threw away the flexibility of having a smaller payment with the option of paying extra principal when & if you chose to.
Much better to have a 30-yr note with $988 payment than a 15-yr note with a $1530 payment. You can always pay $1530 instead of the required $988. But you can't pay $988 when the note requires $1530.
Of course, the smartest thing to do is take the 30 yr, with its $988 payment and put the extra $542 in a savings/investment account. By now you'd have around $59,000 in that account and no worries about having the money to make the mortgage payments in case of unexpected occurrances ... like getting laid off.
Don't compound that mistake be making another two mistakes.
The 1st mistake is paying it off with money you need to keep invested to fund your retirement.
The 2nd mistake is paying it off from a retirement account, 401K/IRA. That itself is two mistakes:
The 1st is using a retirement account for non-retirement expenses.
The 2nd is the withdrawal is taxed as ordinary income, and probably 25% will go to income tax. Just because you don't have the 10% penalty, doesn't mean that it isn't a lump of ordinary income.
What you probably should do would be the opposite -- get a new 30-yr mortgage and take cash out, put that cash into a well diversified investment account.
People talk about the Sleep Well At Night (SWAN) financial situation, but there are really 2 different SWAN propositions.
1) SWAN knowing that you could pay off the mortgage without seriously reducing your investment accounts. And your investment returns are much larger (on average) than the mortgage payment.
2) SWAN because you are unknowingly taking on a much larger financial risk than you think.
If you don't know with absolute certainty that you are in category 1, you are in category 2. People who are agonizing about whether or not to pay off their mortgage are almost always in category 2.