The quietest and most durable fans will have a large fan area and likely a motor that drives the fan using a belt.
In some parts of the country whole-house fans can be useful. Remember that comfort is a function of both temperature and humidity, and it's unlikely you'll be comfortable if you are bringing in cool air that is saturated with moisture (typical in the mornings in much of the country much of the time).
When you run the A/C it not only cools the air but removes moisture, which is usually a plus.
The fans are probably most useful where daily temps have a wide swing and where the humidity is low. I don't like the feeling of a breezy house, so I wouldn't install one. But, I don't care for ceiling fans, either, and I know I'm in the minority. Bld999's suggestion of using an external air inlet and your HVAC ducts to distribute the incoming air throughout the house is a good idea (reduces drafts, more even temperatures), and in some situations it might even be possible to have this air go through your regular furnace filter to remove some of the pollen.
If you are doing this to save money, compute the payback period. It would probably be long in most areas (due to the fairly limited number of hours that outside temp/humidity are more agreeable than inside temp/humidity). The same money spent on attic insulation would boost comfort and reduce utility bills winter and summer, require no maintenance, use no electricity, and reduce noise rather than increase it.
It's another "thing" and requires manual intervention (turn off the HVAC, open windows, turn on the fan).
Also, you'll be depressurizing your building envelope when the whole-house fan is turned on. If you've got enough incoming air it may not be a significant issue, but if you're depending on the gaps and holes of the house it could be a different story. The depressurization may cause problems for combustion appliances (furnace, gas/oil clothes dryer, gas water heater, etc) and result in CO entering your home. It probably won't be a safety factor as long as the fan is pulling a lot of fresh air into the home, but if the fan is on low it could cause an indoor air quality problem in some circumstances.
The hillbilly alternative: A box fan propped up in the window of a little-used room to blow air out, and other windows opened in selected rooms of the house. 20 bucks, and if it proves its worth you could go with a more permanent arrangement.