I think the issue here is what does it means to be "like the past"? There are multiple ways of defining "like" that could lead to widely different forecasts.
For example, consider the following two approaches
(1) You believe that the future stock returns will be like the equally weighted average of returns in the past. So to create your expectation you take a simple average and use that as your forecast.
(2) You believe that future stock returns will be like past years when the market had similar Schiller P/E values (or whatever market condition you think is important). So to create your expectation you take a weighted average (giving higher weight to past years with similar Schiller PE) and use that.
In both cases, we are trying set expectations based on the idea that the future will be like the past. It's just that the definition of "like the past" is a little different in each case.
In some cases like with Schiller PE there's a mountain of evidence that this is a real effect and not some spurious result. Other methods (like using the butter production in bangladesh to predict S&P 500) may have more limited support.
The people who make models to forecast stock returns are all basing it, directly or indirectly, on past data. The model might produce different predictions than the historical average but that's because current conditions are not historically average.
I will not be happy either with such low rates of return. But forecasting is extremely difficult and even the best models (which tend to be pessimistic now) do not exclude good outcomes when you look at the range of expected results instead of a point estimate.
+1 Excellent post.
We of course have no way of knowing the future, and anything we do know we would have to assume is pretty much already priced in. So we base all of our models on what we know of the past. Other than guessing, that is all there is. We just pick and choose what part of that past history we want to emphasize and what parts we want to minimize. Our assumptions control the model, so in some cases, why do the model, we already know from our assumptions how it will turn out.
One reason I like historical estimates like FIRECalc is that there are so few assumptions made, just cranks out how well we would have done in the past, to answer a simple question - are we anywhere near reasonable.
As an individual, I don't see why we want to spend any time trying to predict an unknowable future rate of return. Certainly we cannot forecast sequence of returns, and that could prove a far greater problem for us.
It is not that I am against modelling, I love it and have done a lot, stochastic, ARIMA, multivariate, etc. I just have a hard time figuring out the point of doing it to figure a SWR. We put in our assumptions (based on some info from the past) and churn out results. How it changes our estimated SWR, other than what we can already get from simplistic FIRECalc-like estimates, I haven't a clue. We are just pretending we know more than is possible.
Now if you are a pension fund, well things are different, and you have a whole set of additional problems, modelling is a must. But for individuals, not so much.