I missed this thread the first time, but I had a similar situation last year and we decided to move.
At 37/41 DH realized that there was no reason for us to keep living in the suburbs in a good school district when we're not going to have kids. We were spending 3-4 evenings a week travelling to another neighborhood where all our favorite bars, restaurants and friends were.
We started looking for a house, and after 2 offers that we're glad they didn't work out, we found our dream house. We doubled our mortgage debt, refinanced our old house to get a bigger down payment and moved. We're renting our old house out, since property taxes are tied to when you bought your house in CA, the property taxes are so cheap it's worth it to keep it. It was a hard choice, because we went from 10-15 years from paying off our house to having 2 30 year mortgages, but someday the rental income should cover property taxes for both houses.
We've been extremely happy with our move. Our quality of life has increased, our circle of friends has expanded, and we walk to bars and dinner all the time. We didn't even move our cars this weekend.
Our commute is actually slightly worse than before, but now we don't have to commute for fun. We have a house that fits our needs better (great room instead of small separate living and family rooms), is actually bigger, but has a much more reasonable yard for what we need.
I'd think about your quality of life. Do you want to live in a city? Are you ok with closer neighbors and possibly higher street traffic? Does your wife want to change jobs, or would you just be trading your commute for hers?
My biggest problem now is that I like my house so much I don't want to go to the office anymore.