I have most of our money at Vanguard. Safety is the only real reason I'd spread our money, and I have been considering moving all our tax deferred and tax free accounts to Fidelity (which would be substantial) - this (below) is the only thing that gives me some reservations...
Well Vanguard certainly touts their unique ownership structure (and uses it as an excuse for their lousy customer service and laughably bad website), but they are anything but transparent with their fund shareholders-cum-"owners" about pretty obvious stuff like executive compensation, voting on shareholder issues and so on. So the "you own us" thing doesn't translate into any real-world advantages (and I say this with regrets, as both a great admirer of Jack Bogle and as someone who invested only at Vanguard for many years and would have preferred to continue to do so).
We still have mostly Vanguard funds and ETFs but long since moved custody over to Schwab, which has vastly superior customer service, an excellent website, a bank that pays decent rates AND offers a debit card that reimburses ATM fees worldwide and - last not least - has real brick-and-mortar offices that DW (who is far less interested in investing but likely to outlive me) can visit at any time for help. Vanguard as the only one of the "big three" without physical offices has no excuse for not being excellent at the few things they do offer, but IMHO the only thing they do well is run mutual funds and ETFs and one need not hold those funds at Vanguard.
I think Fido and Schwab are equally good choices to diversify custody for a Vanguard investor. We also appreciate the value of having immediate local access to cash at a credit union. On the other hand we liquidated our holdings at Treasury Direct after experiencing their buggy website and reading numerous nightmare stories about surviving spouses trying to access funds held there. Easy enough to buy T Bills at auction at Schwab or Fido and set them to auto-roll on the cash side (a "roll-your-own" Treasury MM fund with a 0% ER), while buying regular bonds or TIPS is much easier at a brokerage than through TD.