I retired three days after I could take my pension. There was no benefit to staying so I left. I did my victory lap during the last few months as I slowly gave away my projects.
Sounds like you aren't quite ready to retire yet. Even though you feel you are FI, you seem reluctant to leave. Have you considered easing into it. Maybe ask for reduced hours or if that's not an option, find a part-time job or some contract work and semi-retire. You won't lose your entire income stream and the security. You will also use less (or none depending on your new job) of your savings. You will also get to begin getting used to living on less money without totally relying on your savings.
Kind of the best of both worlds. You can keep an income, get used to your new retirement income, stretch and maybe even increase your savings. Maybe you can find a job that suits you better, or work for yourself. If you were 60, I'd say just do it, but at 40 semi-retirement seems like a good fit.
Retired July 4th, 2010 at age 43
Trout Bum, Writer, Full-Time Dad and Husband