I switched from USAA to Vanguard for this reason. In my IRA, I sold the USAA S&P 500 fund and bought VTSAX. In my taxable account, I just moved shares in kind rather than realize gains. I'd rather pay the smaller fee (0.25%) than pay capital gains taxes this year. I'm sure at some point it would be worth cashing out and buying lower fee funds, but I'm also too lazy to do that much math with less than 10% (and shrinking) of my portfolio. Considering that .25% is the highest fee I pay in my accounts, and my overall is <.1%, I'm not worried about it.
So, the answer, as always, is "it depends." But either way, I'd stop reinvesting dividends or adding new funds to TRP fund.
"So we beat to our own drummer in the sun;
We ask for nobody's permission to run.
I just wanna live in a world like that;
Now I'm gonna live in a world like that!" - World Like That, O.A.R.