If you have failed to manage your career so that earning a living and saving for RE means spending your time in pain and aggravation, it's hard to think of the possibility of RE'ing "too early." OTOH, some folks are very successful in being able to earn money not only painlessly but by participating in activities they actually enjoy. Those A+ career managers might indeed retire "too early."
It's an individual thing and 100% dependent on your ability to manage your life so that "earning" and spending your time engaged in enjoyable activities are the same thing.
On this basis, I guess I have to say I retired at exactly the right time (maybe a couple of weeks too late at the worst). I was enjoying what I was doing and getting paid bushels of money to do it. I'd long since gained FI. When the boss said, now you're going to be doing XYZ (something I hated), I said "No I'm not. I'm retiring." Truth be told, I DID, in fact, retire FROM something rather than TO something. Still, it has worked out well and I'm more than satisfied.
I suppose if we're honest (at least with ourselves) we would admit that there are some things we miss about our w*rking days. An obvious one for (probably) many, if not most, would be the social interaction. I discovered a surprise "missed thing" about my w*rking. days. I used to travel once or twice a year for Megacorp. I got to stay at nice hotels, eat good meals (on Megacorp), have the freedom (not just from DW and kids, but also from regular duties and responsibilities) to roam about and discover new things (again at Megacorp expense). Didn't miss it until it was gone. Yes, I still travel, but there is a different dimension to it. In the old days it was a "mission". Now, it's an excursion. We could argue about which is better, but I do miss it, god help me.
Still, I contend that my ER date was, for all practical purposes, right on time. Not too early and not too late. Never really regretted it, though I do miss some things occasionally. YMMV