Your situation sounds like ours. DH and I had just started to think about ER about 5 years ago. I managed to pay off cc and auto loan debt in a couple years, and I streamlined our budget, but our total savings were only about 30k. Then we inherited about 700K (while losing 3 of our parents over a two year period, a most traumatic and heartbreaking experience for us.)
We decided that it made sense for us to pay off our $106k mortgage. This enabled me to cut back my work to 20 hours per week. We have a child who has a learning disability. I am now able to volunteer at her school and contribute the extra "momming" she needs.
We invested about $300 and about $300k is tied up in real estate (other than our primary residence). For now we are planning to work until we qualify to pick up our full public employee pensions. That will be about 7 more years for my hub and about 9 for me. We will both be in our late 50's at retirement, but I still consider it early retirement, considering that otherwise we would work into our mid-sixties.
Factors that influenced us to work on for several more years: I get a discount on college tuition for both my kids if I continue to work in my current position. We get full health care coverage/prescription coverage/vision/dental through my husband's job, and given our daughter's condition and our health (I have asthma, my hub has high blood pressure), we cannot afford to buy health care coverage outside of our jobs (even with nearly 1,000,000 of net worth, counting our home).
I found that cutting back to part time helped me get adjust to the idea of working longer. My hub is not too happy with his job, but he is trying to stick it out. We will evaluate our situation when he turns 55; perhaps he will be able to retire, draw a pension, and find a half time job wiith some bennies (although those are rare.)
I think problem for us is adjusting to a "saver" lifestyle from a "spender" lifestyle. We were just getting used to socking away a few extra dollars per month while living more frugally, when sudden wealth "struck". We have managed to live on our income, but I'd like to improve our savings rate.
And having said that, I really, really love travel and I am planning a vacation in Mexico this summer. I figure it fits into our plan if we pay for it in cash from our work income, which we can easily do. However, I accept that this is rationalizing a very large expenditure!