I'm an AL member, it is marginally useful. It's hard to tell if a contractor is good based on the reviews: How many satisfied customers are not writing anything? If there are a slew of negative reviews, is it just because a particular contractor is very big (more customers=likely more unsatisfied customers). I have the feeling it works best in larger metro areas. Still, it's better than nothing, so I've kept my membership. When I get quotes I let the bidders know I found them on AL, and I subtly remind them again that I'm a member when they start doing the work--maybe they'll take a little extra time to do the job right if they want a positive write up from me.
One thing that bugs me: AL calls me several times a year and asks about contractors I've used (based on searches I've done on their site). They want customers to write reviews, or at least to give some feedback on various businesses when they call. I find this to be intrusive. The calls are always very friendly, but its a minor annoyance.