Well, I'm not sure what people you are "seeing" talking about this, but it won't be most of the people here.
Many people use the NRA from SS as their retirement age because they don't have enough money to live on without SS, so they can't retire before then.
You say, "I can see it as a age to keep working and start collecting SS but not a retirement age."
This sentence doesn't make sense to me. If you working, you are not retired. I don't get the distinction you are trying to make here. Yes, you can collect SS while working, but if you make too much, you will be penalized.
I don't think you are missing anything except the fact that the majority of the population can't retire early because they don't have the money, and will be relying on SS to fund a significant portion of their retirement.