Actually, after further reading, the line separating an employee from an independent contractor is a bit more blurry than I thought: See how the IRS determines whether you are an employee or an independent contractor.
Independent Contractors vs. Employees
You have to at least demonstrate a certain level of independence from your client but it does not say anything about the need to have several clients. But a lawyer friend of mine said that the IRS could look suspiciously at your case if, as an independent contractor, you have only one client. But apparently, in itself, it is not against the law if you can prove you are independent enough from your client.