I agree with the sentiment that being mortgage free by retirement sounds appealing. I've broken down my plan into phases, of which mortgage payoff is last on the list.
1. Pay off student loans (done)
2. Build up nest egg for full retirement at 59.5 in tax deferred accounts. (in progress...)
3. Next, build up taxable account to cover gap between ER and age 59.5 (also in progress...)
4. If practical, pay off mortgage as last step to reduce fixed expenses.
5. Begin living off of investment SWR while still working to test the system while "slowly backing out" of the jobs.