I was bicycling there about 8 years ago. It's a wonderful place to visit, with diverse sights in a relatively compact country. In 3 or 4 weeks, you can pretty much see it all.
My list of must-sees include the following:
Sigiriya - ruins of an ancient palace built on top of a massive rock. If you have vertigo, you probably won't want to climb up the stairs on the side of the rock, however. It's one of the coolest places I've ever visited. Great views on top of the huge rock. Wikipedia has a photo:
Sigiriya - Wikipedia, the free encyclopedia
Horton Plains National Park - I hired a driver for half a day to go there and back from Nurawa Eliya, and to get there before sunrise. Nice easy hike through cloud forest, to a waterfall, and to Land's End for the stunning view south.
Climb up by the ancient British-built train up through the pretty tea plantations into the mountains. Great scenery, atmospheric train, and even wonderful music by a local musician who busked in my train. The temperature is delightful at altitude compared to everywhere else in the country.
Anuradapura - hire a bike to see the spread out ruins
Beaches on the south coast - there were still many signs of the tsunami when I was there.
Galle - pretty walled town
Kandy - several interesting sights. Not far from town, there was a superb botanic garden if that interests you.
Polonnaruwa - I ran out of time and didn't go, but it sounds worthwhile. Also, it's supposed to be one of the best places to see wild elephants.
Elephant Orphanage - between Colombo & Kandy. There were also spice farms around there.
Cave paintings at Dambulla were somewhat interesting.
I didn't go to Colombo at all, which doesn't sound like it has many attractions. I spent my first and last night in Negombo, a pleasant beach town just north of the airport. I also didn't have time to really see Yala West National Park, which sounds nice.
I loved the food, which is quite spicy. "Rice and curry" is a varying mix of different vegetable dishes, dal (spicy and made with coconut milk), rice, and usually a small amount of either chicken or fish. Try rice hoppers for breakfast. Much better than the "western" breakfasts, IMO. Stop in the bakeries. There are some interesting things, both sweet and savory. There aren't too many restaurants outside of Kandy, and presumably Colombo. I typically ate dinner in the guest houses. There seems to be pretty good hygiene. I didn't get the slightest bit sick, nor did I meet anyone who got sick there.
People are friendly, and they don't hassle you too much (unlike in India, I've been told.) Enough people speak English to make it easy to get around. Road signs are usually in English. I met an American who hired a driver during his stay. He said it was extremely inexpensive.
I was there in Jan/Feb, which is normally a drier time in most of the island, except for the east coast, which normally has a monsoon period at that time.