Habitat affiliates are very individual, and most are only as good as the quality of the staff. I worked for the 3rd largest in the country, building more than 25 houses a year. It was run like a business, for the most part. That appealed to me, as I'm not so much into the warm and fuzzy, but poorly run, type of nonprofit.
The program itself is sound, and the national folks in Americus take a big bite out of the affiliate's revenue, think franchise.
The guy they booted as CEO (Millard Fuller) was a Perv, though, no doubt about it. The stories we heard through the grapevine!
Depending on the rules of your particular Habitat affiliate, you can become disillusioned by the way applicants are selected. I remember thinking that we were really punishing two parent, both working folks, since the income max was so low. I knew of several homeowners who gamed the system by booting out Dad in order to qualify, then once the house was built, having him move back in.
Upside: I really enjoyed working with the homeowners in doing their sweat equity. I also liked meeting the volunteers that came through, folks that were really committed to doing some good in their spare time. We were positioned in a rural community, adjacent to very wealthy resort areas and as a result had gobs of money flowing in.
Downside: the (to me) relatively oppressive Christian leadership. I remember the boss coming into our place and throwing away donated books if they had any reference to Muslim or Pagan religions. And he told anti-Catholic jokes. But that was not "party-line" stuff, and no one besides the boss was like that.
I think your mileage will vary. If your local affiliate is a good one, well managed and run for volunteers to get the most out of the experience, you will have a great time. If it isn't, then I'd say find another non-p to support. I can say that the concept is good, and it was gratifying to work in a helping environment.