Actually, I want to fire many of those. The businesses that the govt have to audit will be much lower in number. In 2003, the IRS had 143 Million returns (130 individual returns, plus others were corporate, partnership, trusts, etc)
http://www.fairtaxvolunteer.org/smart/WaysandMeansRebuttal.pdf page 9
With fairtax, all you need to worry about will be businesses. Surely, out of 130 M individual returns in 2003, some may be businesses (sole proprietorship, etc), but I don't think it will be a large percentage. Most trusts will cease to exist; there's no purpose for them. So, conservatively my estimate is that there would only be about 15-20 million tax paying/forwarding entities you'll need to worry about. If you fire 50% of IRS auditors, your audit rate will still be about 3%, about 4 times 0.77%.
Plus, sales tax will be pretty hard, if not harder than income tax, to cheat. Let's say you run a barber shop, you can't cheat your sales volume, IRS (then may be called fairtax enforcer) can just look at the parking lot camera and count your customers. If you're a chiropractor, you have all those insurance claims that will keep you in check. How about Mr. Handyman who takes cash to fix your toilet? I'll say there's no difference. If he wants to cheat now, he doesn't report that cash income. If he wants to cheat under fairtax, he either doesn't collect it or doesn't report it. No difference.
The govt can even discourage sales tax cheat by having a hotline that if you think someone is not reporting sales tax, you can be the whistleblower and can have a piece of that extra collection pie. It would not be easy to cheat, at least not easier than the current convoluted system where honest citizens or even IRS folks don't know what is cheating and what's not.