Funny, I had the opposite impression. Lived and w*rked in Brazil for several years and mastered the language without much trouble.
What I liked was that a Portuguese speaker can easily understand Spanish, while those from the surrounding countries struggle mightily with spoken Portuguese. Written language is OK for reading, but a Brazilian speaker, especially with some regional accents, is nearly incomprehensible to an Argentine. It's kind of like a one-way mirror.
I’ve long had the feeling that the incomprehension was, at least in part, deliberate, but otherwise agree totally with your view.
As for the OP question, one should spend enough to meet ones objective, but not more than one can afford to lose. There are risks to owning property abroad and retiring abroad, and even in a stable advanced economy unexpected things happen.