I think the WSJ article and this thread is somewhat misrepresenting Professor Wurgler
paper.
Index funds aren't causing increase volatility except indirectly in that traders have more derivative products to use.
The problems are related to that increased correlation among different asset classes in general and among the S&P500 stocks in particular. Stock prices move in a more herd like fashion now than the have in the past. This makes it harder for investors to achieve a diversified portfolio. There is hardly much of a point to having small cap, large cap, growth, value, international stocks/fund etc. when the all move the same direction all the time.
The second point is particular to the S&P 500, the paper discusses the premium that occurs when a stock gets included in the S&P500 (20-25 changes are made per year).
His data showed that the stocks in included in the S&P500 may get as much 30% price premium simply by being included in the index. Stocks jump immediately when are included by 5-10% and fall as much as 10-20% when the drop out of the S&P and overtime this over performance continuing. Considering that up until two years ago, a big companies such as Buffett's Berkshire weren't in the S&P there are some problems with the index. But the bigger problem is that as whole the S&P500 has (according the Wurgler) accounts for 78% of the market cap of the US stock market if that is overvalued by 30% we should probably be seeing the S&P only account for 60%. This mean that capital costs for the ~10,000 (generally smaller) companies not in the index are paying too much for capital.
One of the dangers I see is that we are letting Standards and Poor pick the winners and losers by who the included in the index, and given their crappy track record in everything from evaluating mortgage backed securities to municipal bonds I am not thrilled about them taking the place of a real market.
"Markets work best when people think and act independently, not all together",Mr. Sullivan says
This is probably the most subtle argument in the paper. If we assume that valuing stock is actually a skill, and people other than Warren Buffett, are good at it that we need to let them be successful. The only way indexers can enjoy a free ride is because there are active investor who figure out which companies are the next Enron, Citi, Kodak, and the next Google, Apple, Walmart. If the herd mentality of index funds become more dominate, then there is a risk that good and bad manager/companies are treated pretty much the same way, sincenot enough people are acting independently.
Wurgler view is that because of the rise of indexing it is increasingly difficult for active manager to differentiate themselves it is worth reading the paper to understand why.
The argument against indexing is similar to the
paradox of thrift. As individuals it is great that we save, but during a recession if everybody saves then overall demand goes down making the economy worse. We have seen this during this recession and increase government spending has not been able to counter act the impact of increased saving.
The same thing is true for indexing. It is great for individual to use indexes, but if everybody switches to an index funds than we are collectively screwed.