Your situation is similar to mine except that I am single. I retired a few years ago, and I'm still not even 40 years old. My portfolio high water mark was 1.4m, and today it's threatening to dip below the magic 1m number. I've always felt that living on the 50k or so SWR was difficult, but very worthwhile. If I had dependents it would have been completely out of the question for me.
The coolest thing about doing this while we are young is that we really do have the chance to start over if necessary. I'm thinking eventually I would like to get some kind of freelancy or entrepreneurial work going, and this economic downturn has increased those inclinations.
I am certainly happy that I took the chance while I had it... financially spending that money at a local market top was actually the best possible scenario. Going forward I have lots of options: buckle down and frugal my way through, get a job and earn my way up, or just wait a few more years and see how things look then; we might be right back on track. It's certainly a better outlook than the folks living paycheck to paycheck, whose options for dealing with the economy include working harder longer, and, well, not much else.