We did what you are proposing and so far we are very happy with our decision. Our new house is probably worth 300-400K more than the one we sold, our home comprises a much larger percentage of our net worth than yours, and we still have a mortgage (pensions cover our expenses, including the mortgage). Our CA property taxes did double, but since our beach house was a fixer upper and we remodeled before moving in, it could have been much worse.
We would not have predicted just how much moving has added to our quality of life. For one, we are in a smaller city. Neighbors are nice. We spend more time outside - a lot more. We get more exercise. We take walks together on the ocean bluffs. And we have a great house - not oceanfront or even ocean view, but close enough to get there anytime we want.
OP, I say go for it.
We would not have predicted just how much moving has added to our quality of life. For one, we are in a smaller city. Neighbors are nice. We spend more time outside - a lot more. We get more exercise. We take walks together on the ocean bluffs. And we have a great house - not oceanfront or even ocean view, but close enough to get there anytime we want.
OP, I say go for it.