utility of monte carlo simulations

[Travelwanted]

I haven't found any recent posts discussing this.

I am aware of the complexity and numerous calculations run by these simulations. As such I expect a bit of variability in the results. However, the variable is so great as to not be useful as a tool in planning.

My typical calculations all give me 100% success rate with FIREcalc using historical data. But, when I run the monte carlo simulations, I get between a 50 - 100% success rate. Clearly this is not helpful.

Others getting the same wide distribution in results? And if so, why is it considered a useful tool?

 

[ERhoosier]

IMHO The differences prob relate to different assumptions in any given simulation. Typical Monte Carlo may include simulations which fall outside of historical data (i.e. may include "this has never happened before" scenarios). Depending upon the underlying assumptions used and level of complexity of calculations, not all Monte Carlo simulations yield the same results.
https://en.wikipedia.org/wiki/Monte_Carlo_methods_in_finance

IMHO- They can be a useful tool in appreciating just how uncertain the financial future may be. And that's NOT including variables in socio-political dynamics.

As the old saying goes- Ya pays yer money and ya takes yer chances

 

[REWahoo]

I never use the Monte Carlo option on FIRECalc. That totally bypasses the calculator's key attribute - using actual market history to determine a success rate.

 

[ggbutcher]

Travelwanted,

The historical data is limited to, well, history. One could say it's more relevant, as it reflects real market behavior, but you know the old saw, "historical performance is not an indicator of future returns..."

Monte Carlo simulation tests a wider space of possibilities. I don't know the specifics of FireCalc, but the MC I know from work takes repeated "draws" on statistical distributions of various parameters to explore the range of possible outcomes.

Off the top of my head, I would say your FireCalc results reflect a plan with a solid foundation, but you might want to consider your exposure to 'outlier' outcomes.

 

[daylatedollarshort]

The historical data is limited to, well, history. One could say it's more relevant, as it reflects real market behavior, but you know the old saw, "historical performance is not an indicator of future returns..."

I have often wondered if Firecalc had been around just before the great depression what ER success rates it would have shown based on market history up to that point in time.

 

[clifp]

I haven't found any recent posts discussing this.

I am aware of the complexity and numerous calculations run by these simulations. As such I expect a bit of variability in the results. However, the variable is so great as to not be useful as a tool in planning.

My typical calculations all give me 100% success rate with FIREcalc using historical data. But, when I run the monte carlo simulations, I get between a 50 - 100% success rate. Clearly this is not helpful.

Others getting the same wide distribution in results? And if so, why is it considered a useful tool?

I think Monte Carlo simulations I have seen for retirement simulations are of virtually no value. The fundamental assumption of Monte Carlo simulation is that the returns of asset class are uniformly distributed and that next years results are independent of last years results.

If you are simulating coin flips, cards, dice rolls, or maybe even made goals in basketball this is probably a good assumption. I think it is a horrible for simulations of real assets. Does anybody think the fact that stocks were up 32% last year and 16% in 2012 has no impact on the performance of stocks this year. Or that fact real estate prices fell 75% from their peaks in places like Vegas, Florida, and parts of CA, is unrelated to the healthy gains the places have made in the last 3 or 4 years?

Some simulator allow you put in your own estimates of future returns of asset classes, but I now I am not qualified to make those estimates. I am not sure how much smarter the average retire (or even average user of calculator) is than I am.

I think even worse is you get other wise smart researchers like Wade Pfau who say I don't think stocks are going to do as well in the future run their simulations and advise retirees to hold very little stocks. Stocks are much risker if you don't believe there are cyclical things called bear and bull markets and one follows the other. A retire who has the patience to hold during a bear market can enjoy much higher standard of living than one who simple keeps all bonds or CDs.

FIRECalc can't predict the feature, but there is something comfort knowing that if I withdraw X% with this portfolio I'll have 90-100% chance of not running out money even in the worse economic conditions for the 100+ years.

 

[ERhoosier]

The historical data is limited to, well, history. One could say it's more relevant, as it reflects real market behavior, but you know the old saw, "historical performance is not an indicator of future returns..."

Monte Carlo simulation tests a wider space of possibilities. ...

+1

No one could have used purely historical data to predict the 'stagflation' affecting UK in '60's-70's and US in 70's. As Iain Macleod so famously put it in his 1965 speech in the UK House of Commons-
"We now have the worst of both worlds—not just inflation on the one side or stagnation on the other, but both of them together. We have a sort of “stagflation” situation. And history, in modern terms, is indeed being made."

Concurrent Monte Carlo simulations, but not historical data alone, could have included the possibility of such a financial calamity in predicting FIRE portfolio success for one pulling the trigger in mid-'60's.

 

[REWahoo]

No one could have used purely historical data to predict the 'stagflation' affecting UK in '60's-70's and US in 70's. As Iain Macleod so famously put it in his 1965 speech in the UK House of Commons-
"We now have the worst of both worlds—not just inflation on the one side or stagnation on the other, but both of them together. We have a sort of “stagflation” situation. And history, in modern terms, is indeed being made."

Concurrent Monte Carlo simulations, but not historical data alone, could have included the possibility of such a financial calamity in predicting FIRE portfolio success for one pulling the trigger in mid-'60's.

Taking that thinking to it's logical conclusion, is any nest egg amount large enough to guarantee anyone's future financial well-being? Does a Monte Carlo simulation include an asteroid strike?

 

[ERhoosier]

Taking that thinking to it's logical conclusion, is any nest egg amount large enough to guarantee anyone's future financial well-being? Does a Monte Carlo simulation include an asteroid strike?

Nothing can absolutely guarantee one's financial security. Like your example of an asteroid strike, unpredictable political events could destroy ANY nest egg (e.g. confiscatory wealth tax). Ultimately everyone must decide what their own comfort level is to FIRE.

 

[CaliforniaMan]

I think Monte Carlo simulations I have seen for retirement simulations are of virtually no value. The fundamental assumption of Monte Carlo simulation is that the returns of asset class are uniformly distributed and that next years results are independent of last years results...

I am afraid I have to agree. Actually I am a big fan of Monte Carlo techniques, having gotten a Masters degree creating a one some four decades ago. But they have what I think are insurmountable flaws when used in retirement planning.

Two basic problems. The first is what distribution to draw from. If you look at different time spans the distributions are different. Is there one overarching one, or does it change with time. And how would you know.

Clearly each year's change cannot be completely random. The rate of growth must have some limits. There are only so many people, so much land, so many vehicles per person, so much they can eat or wear, etc. And no matter how low the economy dips, people still eat something, wear something, etc. So large swings have to be compensated for somehow in the future, but how?

Nobody knows the answer to either of these. I assume most Monte Carlo models are based on random samples from a known distribution, possibly even from a distribution of real changes. But it doesn't matter. The distribution has to be affected by the past, but how?

When I have made some simple linearized using on Excel and log normal distributions (I know not optimal) the extremes were far higher and far lower than historical returns or what seemed reasonable, unless...

Taking that thinking to it's logical conclusion, is any nest egg amount large enough to guarantee anyone's future financial well-being? Does a Monte Carlo simulation include an asteroid strike?

There is simply no way to know what is a reasonable distribution from which to draw, so I think we are stuck with historical data.

There is a pretty good explantion of this at: http://www.retirementoptimizer.com/articles/MCArticle.pdf

 

[aim-high]

A job doesn't guarantee financial security either. No difference.

 

[Animorph]

I think Monte Carlo simulations I have seen for retirement simulations are of virtually no value. The fundamental assumption of Monte Carlo simulation is that the returns of asset class are uniformly distributed and that next years results are independent of last years results.

If you are simulating coin flips, cards, dice rolls, or maybe even made goals in basketball this is probably a good assumption. I think it is a horrible for simulations of real assets. Does anybody think the fact that stocks were up 32% last year and 16% in 2012 has no impact on the performance of stocks this year. Or that fact real estate prices fell 75% from their peaks in places like Vegas, Florida, and parts of CA, is unrelated to the healthy gains the places have made in the last 3 or 4 years?

Some simulator allow you put in your own estimates of future returns of asset classes, but I now I am not qualified to make those estimates. I am not sure how much smarter the average retire (or even average user of calculator) is than I am.

I think even worse is you get other wise smart researchers like Wade Pfau who say I don't think stocks are going to do as well in the future run their simulations and advise retirees to hold very little stocks. Stocks are much risker if you don't believe there are cyclical things called bear and bull markets and one follows the other. A retire who has the patience to hold during a bear market can enjoy much higher standard of living than one who simple keeps all bonds or CDs.

FIRECalc can't predict the feature, but there is something comfort knowing that if I withdraw X% with this portfolio I'll have 90-100% chance of not running out money even in the worse economic conditions for the 100+ years.

+1.

Monte Carlo simulations use assumptions and compromises to "match" historical statistics. And they're mostly all different. And none that I used had anything like reversion to the mean.

 

[Travelwanted]

Thanks for the great replies.

I completely understand the notion that there are no guarantees. But admittedly seeing 100% in FIREcalc makes me feel pretty solid. I'm trying to reassure myself about RE in an uncertain world. (RE 2015)

I just wanted to see if others see MC simulations as I do: interesting but not helpful for planning. No one can plan for every scenario. If we fail with our conservative numbers I think our society will be facing far larger issues than we care to pontificate about.

 

[Totoro]

Thanks for the great replies.

I completely understand the notion that there are no guarantees. But admittedly seeing 100% in FIREcalc makes me feel pretty solid. I'm trying to reassure myself about RE in an uncertain world. (RE 2015)

I just wanted to see if others see MC simulations as I do: interesting but not helpful for planning. No one can plan for every scenario. If we fail with our conservative numbers I think our society will be facing far larger issues than we care to pontificate about.

As with everything in predicting the future, it is not so much about the results but the assumptions you rely on.

With firecalc, you assume that future returns will look similar to past returns. With a monte carlo simulation, you substitute that with a statistical distribution. Both are wrong in different ways, so I think they are still useful. I use other methods as well. The aggregate perspective gives you a picture of the risk, which you cannot fully eliminate as you point out. I prefer multiple tools and then draw conclusions, it makes the end product more robust, and usually you learn more about the assumptions you rely on.

In the end however, for long term returns, especially if you rely on indexing for the stock portfolio, your returns will depend on world economic growth and what share of that wealth will go the owners.

Both factors are known unknowns

 

[photoguy]

With firecalc, you assume that future returns will look similar to past returns. With a monte carlo simulation, you substitute that with a statistical distribution. Both are wrong in different ways, so I think they are still useful. I use other methods as well. The aggregate perspective gives you a picture of the risk, which you cannot fully eliminate as you point out. I prefer multiple tools and then draw conclusions, it makes the end product more robust, and usually you learn more about the assumptions you rely on.

+1000.

In stats there is a notion that "all models are wrong", but that doesn't mean they can't provide useful information.

I think one key thing for Travelwanted to understand is why the MC methods are giving lower success rates then Firecalc. One driver behind the higher failure rate is that many MC studies use expected returns lower than in seen in the US historical past. There are good reasons for believing this will be true going forward, but this is something each person has to decide for themselves.

 

[Travelwanted]

I think one key thing for Travelwanted to understand is why the MC methods are giving lower success rates then Firecalc. One driver behind the higher failure rate is that many MC studies use expected returns lower than in seen in the US historical past. There are good reasons for believing this will be true going forward, but this is something each person has to decide for themselves.

I don't agree or disagree with the conjecture that returns will be lower going forward. I don't know. But the idea the MC is adjusting for this seems incorrect. I say that because of the significant variability in the results. Many simulations showed 95-100%. However, a couple showed 50%. In one run I had a chance to become a billionaire...seriously.

I realize this is telling me it is highly unlikely except under catastrophic conditions that we will not succeed. But I do not think it implies lower returns moving forward. I think it runs random and sometimes extreme variable and gives outputs resulting from henceforth unseen economic collapse.

 

[photoguy]

TW - I'm not sure which MC simulations you ran and it may not be a factor in those specific results. However, there are several MC studies (e.g., by Pfau and others) where they reduce US returns from historical and then not surprisingly end up with much higher failure rates compared with firecalc.

 

[ggbutcher]

My day job is mostly about shaping peoples' heads about risk management and the appropriate use of statistical probability therein. Some of that applies to my retirement thinking, which I will now impose upon you...

With regard to probability, it's hard to get folk off the notion that the little percent number they so carefully computed reflects a precise assessment of the chance something will occur. I've watched hours-long arguments about tenth-of-a-point differences in probabilities - really, folks? You've gotta take all statistical probabilities with a grain of salt.

With regard to risk management, I believe this is the more fruitful area of concern in retirement planning. And, that's not about the risk exposure of some specific investment, it's about spreading our exposure to a variety of investment and income streams, so an undesired outcome in a single one doesn't take the whole pot. Mutual funds help us do that with respect to equity investment, but it's mostly up to the individual to manage that with other sources with more conservative risk-reward tendencies. FWIW, I've spread my income streams among two DB pensions, 401k, and SS. That way, a single bad event shouldn't take me out totally. Okay, asteroid strike...

In that regard, I'd look at Firecalc's MC option as a risk assessment tool. If the numbers out of it are significantly different from the historical tool, you might consider what it may be telling you about risk exposure...

 

[Totoro]

Many simulations showed 95-100%. However, a couple showed 50%. In one run I had a chance to become a billionaire...seriously.

I realize this is telling me it is highly unlikely except under catastrophic conditions that we will not succeed. But I do not think it implies lower returns moving forward. I think it runs random and sometimes extreme variable and gives outputs resulting from henceforth unseen economic collapse.

Depending on your exact parameters, this is correct.

The issue you have is likely caused by two things:

• Your simulation is memory-less. This means that the return of any given year does not depend on the previous year. This is a big difference with Firecalc.
• The distribution you choose is not suited very well to model these things. Probably you have a normal distribution, or maybe even a uniform one?

Both combined mean that it is quite easy for the simulation to spit out a run where you have -50% returns for 10 years in a row, or conversely +100% for 10 years in a row.

This gives you the billion dollars and speedy bankruptcy outcomes.

Please keep in mind that a monte carlo simulation by definition runs random variables. It is up to you to decide how random you want it to be.

 

[dallas27]

MC sims can give ridiculous results. Start with a million, compound for 20 years then spend 35k annually, and you are broke in 2 years. Makes no sense but total randomness can create that scenario where the sp500 can be bought as a penny stock then recover.

With MC I exclude the top and bottom 10% of results and consider them fantasy. With one, attacking aliens are driving the market down, with the other, I am breathing gold plated oxygen to show off my wealth.


Sent from my iPhone using Early Retirement Forum

 

[BBQ-Nut]

MC sims can give ridiculous results. Start with a million, compound for 20 years then spend 35k annually, and you are broke in 2 years. Makes no sense but total randomness can create that scenario where the sp500 can be bought as a penny stock then recover.

With MC I exclude the top and bottom 10% of results and consider them fantasy. With one, attacking aliens are driving the market down, with the other, I am breathing gold plated oxygen to show off my wealth.


Sent from my iPhone using Early Retirement Forum

Brilliant!

Good thing I wasn't sipping something while reading that post!

 

[Totoro]

A 10% chance in becoming Midas The Second .. nice odds!

 

[gcgang]

I feel more comfortable knowing my plan has survived 1500-1800 out of 2000 simulations using data derived from history, than 100% success using one set of actual historical data.

My goal in using MC is to figure out how much I can spend without significant risk of running out of money (>75% success), and without either taking on too much risk or leaving a significantly larger legacy than I intend (<90%).

With the gains the market has delivered in the last few years, my % success has gone above 90%. Now, I have pleasant options to keep my 75-90% range: 1) increase spending; 2) maintain spending, but adopt a more conservative AA. What I am doing is a little bit of both, increasing cash holdings while upping charitable contributions and some discretionary spending.

I almost feel like this is cheating, ending up doing the forbidden market timing by reducing risk now, and being in position to take on more risk when the inevitable correction does occur.

 

[DFA]

I got my retirement plan review (Free) regular, proposed, and MC back from Schwab today; I don't know why I keep doing this, it must be insanity and like Albert said:
Insanity: doing the same thing over and over again and expecting different results.

The regular plans (current and proposed) calculator said we are fine (spend level got pretty high 2X planned at some points), but MC showed that we could be broke in 15 years or very rich at 93.

I guess I will stick with FireCalc, my plan, their current and proposed plan results and use the MC as a guideline to look at if things get really bad.

 

[Travelwanted]

While I like that saying, it wasn't Albert Einstein:

The above quote has been mis-attributed to Albert Einstein, Benjamin Franklin, and Mark Twain. In fact, none of these great minds were responsible for such a convincing, yet blatantly incorrect definition. The first time it actually appeared in print was in a 1981 Narcotics Anonymous text.

I also like the one: the most powerful force in the universe is compound interest. Also attributed to AE. It is uncertain if this is true.