Nords captured it pretty well. And one thing I really can't understand is why anyone would label this subject as 'controversial'? There are two elements, financial and emotional:
FINANCIAL: From everything I've seen, and from what others have reported, there may be a slight positive bias towards keeping the debt - but it depends on your assumptions. I've never seen it make a big difference.
EMOTIONAL: Some people seem to be able to sleep better at night after moving a chuck of money from a diversified portfolio of investments into a single piece of RE. So they should do that if they want, and they can also sleep easier knowing that if they look back at the financial side of things, it probably won't make much difference either way. One caveat (as mentioned by others) is that you want to make sure you maintain sufficient liquidity - that can indeed become a real-life problem.
You've summarized this nicely.
I've spent hours reading the threads regarding paying off the mortgage. By not paying of the mortgage, cash-flow does improve and there seems to be a slight overall advantage. But as you said, it doesn't seem to ever make a big difference.
I should probably start another thread, instead of hijacking this one, but maybe somebody can quickly point out what I'm missing.
If I have 12k/year mortgage payment, at a 4% WR that means I'd need to have 300k in capital to sustain this payment. The initial balance was 190k, but by the pay-off time (let's say 8 years into the mortgage) it would be ~162k. Why would you choose not to pay it off in this case?
And I think I just realized the answer. The WR could be higher, since I'm only dealing with 22 years before this expense is paid-off. Is that right? Are there any other factors that I'm missing?