YES, this can be a big problem, no matter how you try to deal with it.
We lived in Big City, Texas, and spent over four years trying to find the right retirement property in the Hill Country - I personally have zero use for Texas at all, but that is DW home state and she wanted to be close to...