When you say financially it doesn't make sense, do you mean affordability-wise or bang-for-the-buck wise? After all, if retirement isn't meant to enjoy those things you can afford in your latter years, then what's the use of retiring? Read up on the thread, "Blow That Dough" and consider treating your wife to the home she's dreaming of. Might not make financial sense, but if you've been saving for a lifetime for retirement, then now is the time to enjoy those savings. Me personally, I find trips to financially not make sense. Rather stay home and develop my own home into a resort.
Besides, those kids come back for visits with a spouse, grand kids, even their friends some day and won't you be glad you can host them all in comfort? We have 2 sons, married with a total of 6 grand kids. They live within 2 hours, and we are on the way to the mountains for camping, skiing, etc. So they visit fairly regularly. Add that they all stay for Christmas because we have a place large enough to host them all, and I could not be more pleased that we spend money on a larger home than not. Now, when my sister-in-law visits, I kinda wish we didn't have the size, but that's my cue to take an extended fishing trip. Ha!