I don't think they can be an accurate gauge of net worth, but a tax return can at least provide some pieces. However, sometimes that information can be corrupted, twisted, and taken out of context.
For instance, here's a little fiasco I recently went through with my family. My Granddad died back in late 2016. He had three sons. My Dad was living with Granddad, and taking care of him. The other two didn't help out much, and also thought of my Dad as a leech. Nevermind the fact that, without my Dad, Granddad would have gone into assisted living decades ago, and that would have eaten up his net worth most likely, so there would be no inheritance.
Anyway, Dad died in early 2017. When my two uncles were going through some paperwork, they found Granddad's tax returns from awhile back. One of them showed that Granddad made about $7,000 in a money market account, back in 2007 or 2008. Considering what MMs were paying at the time (2017), they assumed Granddad had well over $1M back then, and my Dad somehow managed to make it "disappear".
There's just one little inconsistency. Back in 2007-2008, MMs were paying 5%+. I had one, with Emigrant Direct. To get 7,000 in interest at 5%, you only needed $140,000. I tried explaining this to them, and it went in one ear and out the other.
I guess there are certain things on a tax return that will let people know you're well off, such as how much dividends, capital gains, etc show up. And if there's a big mortgage deduction, that's a sign you at least qualified for an expensive house. So, people could piece together and infer. But I don't think you can actually peg down an actual net worth number from a tax return.