Firecalc shows that financially, a mortgage is slightly better than no mortgage.
Consider a $300K mortgage at 4%, P&I payment is $1432/mo or $17,184/yr. Assume an investment portfolio of $1.3M.
WITH MORTGAGE
$1.3M with $52K withdrawal (4%) has 95% success rate (30 years). Average final balance $2.4M. The withdrawal is increased by inflation. Mortgage payment comes out of the withdrawal, but that payment is fixed. The net spendable will go up over time, because the mortgage payment is fixed.
NO MORTGAGE
If you pay off the mortgage from the portfolio and keep the same net spendable withdrawal, the draw is $34,816 ($52k - $17,184).
When you plug $1M portfolio and $34,816 withdrawal, Firecalc reports 100% success rate. Average final balance $2.3M
SUMMARY
Financially, there is virtually no difference between keeping the mortgage or paying it off. Therefore, the decision to pay it off or not is a matter of personal preference.