There are a number of us here from the San Diego area on this forum. There is so much to do here and so many different type of activities, as Rdub mentions, it really depends on what kind of things you like to do and your activity level. Your 3 hour drive puts you in reach of LA and all you can do there, and the coastal areas in between. I love the ocean, and we have a lot of great beaches, snorkeling at La Jolla cove. In San Diego itself when I have visitors I take them to Old Town (Like Mexico without the fear), and spend a day visiting Little Italy and then walk down to the Star of India sailing ships (and other ships in the museum). Usually we do Sea World one day, and a trip to Julian another day. The deserts are interesting, but I rather avoid them in summer. In the downtown Gaslamp district all kinds of clubs and music. Coronado is nice, has a nice park along the bay, but avoid the famous Coronado Hotel's restaurants. Balboa park is great and has museums and zoo. Our museums are fairly simple, but the zoo is world class, as is the Safari Park, also run by the S.D. Zoological Society. For your weekend trips Disneyland and LA, Hollywood are interesting. Actually I lived in SD area most of my life but only go to see Hollywood when we have out of town guests. But Universal Studios has some interesting rides and tours of where and how movies are made.
But there is really so much to do in San Diego (and the 3 hour side trips) it is difficult choose. Let us know what kind of things you like to do, if you are bringing kids, etc. and I am sure you will get some good feedback. It is a great place to visit or live. Am sure you will have fun.