If you didn't have all your managed funds, would you replace them with just an S&P 500 index fund? If so, by all means compare them to the S&P 500. I might pick a Vanguard target retirement index or a fixed allocation index instead. Though it might be better to compare the portfolio as a whole, not each individual fund.
I have a slice and dice portfolio. I have plenty of actively managed funds in there. But they are serving within specific allocations, and I am comparing them to an index ETF that is appropriate for that allocation slice. I'm OK when a manager adds some foreign stocks to a mostly domestic fund, or EM to a mostly developed countries fund, and I even have a couple of global funds in there. But if they don't come close to the index ETF for that slice, over a good number of years, they're replaced by shares of that index ETF.
If my alternative was a lazy portfolio of 3 index funds, than I'd benchmark my funds against one of those three index funds.
The big point is that sometimes foreign does better than domestic, or growth better than value, small better than large, or EM better than everything else. Funds that specialize in these categories can't be graded against the S&P 500. That tells you nothing about how good those managers are doing. Even if they were closet indexers their fund mandate might not allow them to look anything like the S&P 500. In that case, you'll hopefully have about half the years beating the S&P 500 and half the years losing to the S&P 500. But hardly ever matching the S&P 500.