...and then there is the proverbial .....you are so lucky to be able to retire.
To which my spouse often replies, yes he was lucky to be able to work 10 and 12 hour day, to be able to spend so much time away from home on business, and take to be able to take calls and con calls at unsocial hours because his responsibility covered multiple time zones.
As a military retiree (USN, 28+ years) I often hear, "You military retirees are so lucky to be able retire so young and get such a nice pension." There was a time when I would mention how lucky we were to get to move every 2-4 years, spend time at sea, have overseas duty, have to move kids from one school to another, etc. I've mellowed over the years and now I just smile and say, "Yes, it's very nice."