Join Early Retirement Today
Reply
 
Thread Tools Search this Thread Display Modes
utility of monte carlo simulations
Old 06-27-2014, 06:19 PM   #1
Recycles dryer sheets
 
Join Date: Mar 2014
Location: Islands
Posts: 330
utility of monte carlo simulations

I haven't found any recent posts discussing this.

I am aware of the complexity and numerous calculations run by these simulations. As such I expect a bit of variability in the results. However, the variable is so great as to not be useful as a tool in planning.

My typical calculations all give me 100% success rate with FIREcalc using historical data. But, when I run the monte carlo simulations, I get between a 50 - 100% success rate. Clearly this is not helpful.

Others getting the same wide distribution in results? And if so, why is it considered a useful tool?
__________________

__________________
Travelwanted is offline   Reply With Quote
Join the #1 Early Retirement and Financial Independence Forum Today - It's Totally Free!

Are you planning to be financially independent as early as possible so you can live life on your own terms? Discuss successful investing strategies, asset allocation models, tax strategies and other related topics in our online forum community. Our members range from young folks just starting their journey to financial independence, military retirees and even multimillionaires. No matter where you fit in you'll find that Early-Retirement.org is a great community to join. Best of all it's totally FREE!

You are currently viewing our boards as a guest so you have limited access to our community. Please take the time to register and you will gain a lot of great new features including; the ability to participate in discussions, network with our members, see fewer ads, upload photographs, create a retirement blog, send private messages and so much, much more!

Old 06-27-2014, 06:52 PM   #2
Thinks s/he gets paid by the post
 
Join Date: Feb 2011
Posts: 1,629
IMHO The differences prob relate to different assumptions in any given simulation. Typical Monte Carlo may include simulations which fall outside of historical data (i.e. may include "this has never happened before" scenarios). Depending upon the underlying assumptions used and level of complexity of calculations, not all Monte Carlo simulations yield the same results.
https://en.wikipedia.org/wiki/Monte_...ods_in_finance

IMHO- They can be a useful tool in appreciating just how uncertain the financial future may be. And that's NOT including variables in socio-political dynamics.

As the old saying goes- Ya pays yer money and ya takes yer chances
__________________

__________________
ERhoosier is offline   Reply With Quote
Old 06-27-2014, 06:53 PM   #3
Give me a museum and I'll fill it. (Picasso) Give me a forum ...
REWahoo's Avatar
 
Join Date: Jun 2002
Location: Texas Hill Country
Posts: 42,068
I never use the Monte Carlo option on FIRECalc. That totally bypasses the calculator's key attribute - using actual market history to determine a success rate.
__________________
Numbers is hard

When I hit 70, it hit back

Retired in 2005 at age 58, no pension
REWahoo is offline   Reply With Quote
Old 06-27-2014, 07:13 PM   #4
Recycles dryer sheets
 
Join Date: Jan 2014
Posts: 246
Travelwanted,

The historical data is limited to, well, history. One could say it's more relevant, as it reflects real market behavior, but you know the old saw, "historical performance is not an indicator of future returns..."

Monte Carlo simulation tests a wider space of possibilities. I don't know the specifics of FireCalc, but the MC I know from work takes repeated "draws" on statistical distributions of various parameters to explore the range of possible outcomes.

Off the top of my head, I would say your FireCalc results reflect a plan with a solid foundation, but you might want to consider your exposure to 'outlier' outcomes.
__________________
ggbutcher is offline   Reply With Quote
Old 06-27-2014, 07:19 PM   #5
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
 
Join Date: Feb 2013
Posts: 5,325
Quote:
Originally Posted by ggbutcher View Post
The historical data is limited to, well, history. One could say it's more relevant, as it reflects real market behavior, but you know the old saw, "historical performance is not an indicator of future returns..."
I have often wondered if Firecalc had been around just before the great depression what ER success rates it would have shown based on market history up to that point in time.
__________________
daylatedollarshort is offline   Reply With Quote
Old 06-27-2014, 08:07 PM   #6
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
clifp's Avatar
 
Join Date: Oct 2006
Posts: 7,450
Quote:
Originally Posted by Travelwanted View Post
I haven't found any recent posts discussing this.

I am aware of the complexity and numerous calculations run by these simulations. As such I expect a bit of variability in the results. However, the variable is so great as to not be useful as a tool in planning.

My typical calculations all give me 100% success rate with FIREcalc using historical data. But, when I run the monte carlo simulations, I get between a 50 - 100% success rate. Clearly this is not helpful.

Others getting the same wide distribution in results? And if so, why is it considered a useful tool?

I think Monte Carlo simulations I have seen for retirement simulations are of virtually no value. The fundamental assumption of Monte Carlo simulation is that the returns of asset class are uniformly distributed and that next years results are independent of last years results.

If you are simulating coin flips, cards, dice rolls, or maybe even made goals in basketball this is probably a good assumption. I think it is a horrible for simulations of real assets. Does anybody think the fact that stocks were up 32% last year and 16% in 2012 has no impact on the performance of stocks this year. Or that fact real estate prices fell 75% from their peaks in places like Vegas, Florida, and parts of CA, is unrelated to the healthy gains the places have made in the last 3 or 4 years?

Some simulator allow you put in your own estimates of future returns of asset classes, but I now I am not qualified to make those estimates. I am not sure how much smarter the average retire (or even average user of calculator) is than I am.

I think even worse is you get other wise smart researchers like Wade Pfau who say I don't think stocks are going to do as well in the future run their simulations and advise retirees to hold very little stocks. Stocks are much risker if you don't believe there are cyclical things called bear and bull markets and one follows the other. A retire who has the patience to hold during a bear market can enjoy much higher standard of living than one who simple keeps all bonds or CDs.

FIRECalc can't predict the feature, but there is something comfort knowing that if I withdraw X% with this portfolio I'll have 90-100% chance of not running out money even in the worse economic conditions for the 100+ years.
__________________
clifp is offline   Reply With Quote
Old 06-27-2014, 09:39 PM   #7
Thinks s/he gets paid by the post
 
Join Date: Feb 2011
Posts: 1,629
Quote:
Originally Posted by ggbutcher View Post

The historical data is limited to, well, history. One could say it's more relevant, as it reflects real market behavior, but you know the old saw, "historical performance is not an indicator of future returns..."

Monte Carlo simulation tests a wider space of possibilities. ...
+1

No one could have used purely historical data to predict the 'stagflation' affecting UK in '60's-70's and US in 70's. As Iain Macleod so famously put it in his 1965 speech in the UK House of Commons-
"We now have the worst of both worlds—not just inflation on the one side or stagnation on the other, but both of them together. We have a sort of “stagflation” situation. And history, in modern terms, is indeed being made."

Concurrent Monte Carlo simulations, but not historical data alone, could have included the possibility of such a financial calamity in predicting FIRE portfolio success for one pulling the trigger in mid-'60's.
__________________
ERhoosier is offline   Reply With Quote
Old 06-27-2014, 09:46 PM   #8
Give me a museum and I'll fill it. (Picasso) Give me a forum ...
REWahoo's Avatar
 
Join Date: Jun 2002
Location: Texas Hill Country
Posts: 42,068
Quote:
Originally Posted by ERhoosier View Post
No one could have used purely historical data to predict the 'stagflation' affecting UK in '60's-70's and US in 70's. As Iain Macleod so famously put it in his 1965 speech in the UK House of Commons-
"We now have the worst of both worlds—not just inflation on the one side or stagnation on the other, but both of them together. We have a sort of “stagflation” situation. And history, in modern terms, is indeed being made."

Concurrent Monte Carlo simulations, but not historical data alone, could have included the possibility of such a financial calamity in predicting FIRE portfolio success for one pulling the trigger in mid-'60's.
Taking that thinking to it's logical conclusion, is any nest egg amount large enough to guarantee anyone's future financial well-being? Does a Monte Carlo simulation include an asteroid strike?
__________________
Numbers is hard

When I hit 70, it hit back

Retired in 2005 at age 58, no pension
REWahoo is offline   Reply With Quote
Old 06-27-2014, 09:59 PM   #9
Thinks s/he gets paid by the post
 
Join Date: Feb 2011
Posts: 1,629
Quote:
Originally Posted by REWahoo View Post
Taking that thinking to it's logical conclusion, is any nest egg amount large enough to guarantee anyone's future financial well-being? Does a Monte Carlo simulation include an asteroid strike?
Nothing can absolutely guarantee one's financial security. Like your example of an asteroid strike, unpredictable political events could destroy ANY nest egg (e.g. confiscatory wealth tax). Ultimately everyone must decide what their own comfort level is to FIRE.
__________________
ERhoosier is offline   Reply With Quote
Old 06-27-2014, 10:00 PM   #10
Full time employment: Posting here.
CaliforniaMan's Avatar
 
Join Date: Dec 2013
Location: San Diego
Posts: 846
Quote:
Originally Posted by clifp View Post
I think Monte Carlo simulations I have seen for retirement simulations are of virtually no value. The fundamental assumption of Monte Carlo simulation is that the returns of asset class are uniformly distributed and that next years results are independent of last years results...
I am afraid I have to agree. Actually I am a big fan of Monte Carlo techniques, having gotten a Masters degree creating a one some four decades ago. But they have what I think are insurmountable flaws when used in retirement planning.

Two basic problems. The first is what distribution to draw from. If you look at different time spans the distributions are different. Is there one overarching one, or does it change with time. And how would you know.

Clearly each year's change cannot be completely random. The rate of growth must have some limits. There are only so many people, so much land, so many vehicles per person, so much they can eat or wear, etc. And no matter how low the economy dips, people still eat something, wear something, etc. So large swings have to be compensated for somehow in the future, but how?

Nobody knows the answer to either of these. I assume most Monte Carlo models are based on random samples from a known distribution, possibly even from a distribution of real changes. But it doesn't matter. The distribution has to be affected by the past, but how?

When I have made some simple linearized using on Excel and log normal distributions (I know not optimal) the extremes were far higher and far lower than historical returns or what seemed reasonable, unless...

Quote:
Originally Posted by REWahoo View Post
Taking that thinking to it's logical conclusion, is any nest egg amount large enough to guarantee anyone's future financial well-being? Does a Monte Carlo simulation include an asteroid strike?
There is simply no way to know what is a reasonable distribution from which to draw, so I think we are stuck with historical data.

There is a pretty good explantion of this at: http://www.retirementoptimizer.com/a.../MCArticle.pdf
__________________
Merrily, merrily, merrily, merrily,
Life is but a dream.
CaliforniaMan is offline   Reply With Quote
Old 06-27-2014, 10:01 PM   #11
Recycles dryer sheets
 
Join Date: Aug 2013
Posts: 349
A job doesn't guarantee financial security either. No difference.
__________________
aim-high is offline   Reply With Quote
Old 06-27-2014, 10:55 PM   #12
Thinks s/he gets paid by the post
 
Join Date: Jul 2005
Posts: 3,862
Quote:
Originally Posted by clifp View Post
I think Monte Carlo simulations I have seen for retirement simulations are of virtually no value. The fundamental assumption of Monte Carlo simulation is that the returns of asset class are uniformly distributed and that next years results are independent of last years results.

If you are simulating coin flips, cards, dice rolls, or maybe even made goals in basketball this is probably a good assumption. I think it is a horrible for simulations of real assets. Does anybody think the fact that stocks were up 32% last year and 16% in 2012 has no impact on the performance of stocks this year. Or that fact real estate prices fell 75% from their peaks in places like Vegas, Florida, and parts of CA, is unrelated to the healthy gains the places have made in the last 3 or 4 years?

Some simulator allow you put in your own estimates of future returns of asset classes, but I now I am not qualified to make those estimates. I am not sure how much smarter the average retire (or even average user of calculator) is than I am.

I think even worse is you get other wise smart researchers like Wade Pfau who say I don't think stocks are going to do as well in the future run their simulations and advise retirees to hold very little stocks. Stocks are much risker if you don't believe there are cyclical things called bear and bull markets and one follows the other. A retire who has the patience to hold during a bear market can enjoy much higher standard of living than one who simple keeps all bonds or CDs.

FIRECalc can't predict the feature, but there is something comfort knowing that if I withdraw X% with this portfolio I'll have 90-100% chance of not running out money even in the worse economic conditions for the 100+ years.
+1.

Monte Carlo simulations use assumptions and compromises to "match" historical statistics. And they're mostly all different. And none that I used had anything like reversion to the mean.
__________________
Animorph is offline   Reply With Quote
Old 06-28-2014, 04:37 AM   #13
Recycles dryer sheets
 
Join Date: Mar 2014
Location: Islands
Posts: 330
Thanks for the great replies.

I completely understand the notion that there are no guarantees. But admittedly seeing 100% in FIREcalc makes me feel pretty solid. I'm trying to reassure myself about RE in an uncertain world. (RE 2015)

I just wanted to see if others see MC simulations as I do: interesting but not helpful for planning. No one can plan for every scenario. If we fail with our conservative numbers I think our society will be facing far larger issues than we care to pontificate about.
__________________
Travelwanted is offline   Reply With Quote
Old 06-28-2014, 06:14 AM   #14
Thinks s/he gets paid by the post
 
Join Date: May 2014
Location: Utrecht
Posts: 2,204
Quote:
Originally Posted by Travelwanted View Post
Thanks for the great replies.

I completely understand the notion that there are no guarantees. But admittedly seeing 100% in FIREcalc makes me feel pretty solid. I'm trying to reassure myself about RE in an uncertain world. (RE 2015)

I just wanted to see if others see MC simulations as I do: interesting but not helpful for planning. No one can plan for every scenario. If we fail with our conservative numbers I think our society will be facing far larger issues than we care to pontificate about.
As with everything in predicting the future, it is not so much about the results but the assumptions you rely on.

With firecalc, you assume that future returns will look similar to past returns. With a monte carlo simulation, you substitute that with a statistical distribution. Both are wrong in different ways, so I think they are still useful. I use other methods as well. The aggregate perspective gives you a picture of the risk, which you cannot fully eliminate as you point out. I prefer multiple tools and then draw conclusions, it makes the end product more robust, and usually you learn more about the assumptions you rely on.

In the end however, for long term returns, especially if you rely on indexing for the stock portfolio, your returns will depend on world economic growth and what share of that wealth will go the owners.

Both factors are known unknowns
__________________
Totoro is offline   Reply With Quote
Old 06-28-2014, 07:10 AM   #15
Thinks s/he gets paid by the post
photoguy's Avatar
 
Join Date: Jun 2010
Posts: 2,301
Quote:
Originally Posted by Totoro View Post
With firecalc, you assume that future returns will look similar to past returns. With a monte carlo simulation, you substitute that with a statistical distribution. Both are wrong in different ways, so I think they are still useful. I use other methods as well. The aggregate perspective gives you a picture of the risk, which you cannot fully eliminate as you point out. I prefer multiple tools and then draw conclusions, it makes the end product more robust, and usually you learn more about the assumptions you rely on.
+1000.

In stats there is a notion that "all models are wrong", but that doesn't mean they can't provide useful information.

I think one key thing for Travelwanted to understand is why the MC methods are giving lower success rates then Firecalc. One driver behind the higher failure rate is that many MC studies use expected returns lower than in seen in the US historical past. There are good reasons for believing this will be true going forward, but this is something each person has to decide for themselves.
__________________
photoguy is offline   Reply With Quote
Old 06-28-2014, 08:25 AM   #16
Recycles dryer sheets
 
Join Date: Mar 2014
Location: Islands
Posts: 330
Quote:
Originally Posted by photoguy View Post
I think one key thing for Travelwanted to understand is why the MC methods are giving lower success rates then Firecalc. One driver behind the higher failure rate is that many MC studies use expected returns lower than in seen in the US historical past. There are good reasons for believing this will be true going forward, but this is something each person has to decide for themselves.
I don't agree or disagree with the conjecture that returns will be lower going forward. I don't know. But the idea the MC is adjusting for this seems incorrect. I say that because of the significant variability in the results. Many simulations showed 95-100%. However, a couple showed 50%. In one run I had a chance to become a billionaire...seriously.

I realize this is telling me it is highly unlikely except under catastrophic conditions that we will not succeed. But I do not think it implies lower returns moving forward. I think it runs random and sometimes extreme variable and gives outputs resulting from henceforth unseen economic collapse.
__________________
Travelwanted is offline   Reply With Quote
Old 06-28-2014, 08:41 AM   #17
Thinks s/he gets paid by the post
photoguy's Avatar
 
Join Date: Jun 2010
Posts: 2,301
TW - I'm not sure which MC simulations you ran and it may not be a factor in those specific results. However, there are several MC studies (e.g., by Pfau and others) where they reduce US returns from historical and then not surprisingly end up with much higher failure rates compared with firecalc.
__________________
photoguy is offline   Reply With Quote
Old 06-28-2014, 11:19 AM   #18
Recycles dryer sheets
 
Join Date: Jan 2014
Posts: 246
My day job is mostly about shaping peoples' heads about risk management and the appropriate use of statistical probability therein. Some of that applies to my retirement thinking, which I will now impose upon you...

With regard to probability, it's hard to get folk off the notion that the little percent number they so carefully computed reflects a precise assessment of the chance something will occur. I've watched hours-long arguments about tenth-of-a-point differences in probabilities - really, folks? You've gotta take all statistical probabilities with a grain of salt.

With regard to risk management, I believe this is the more fruitful area of concern in retirement planning. And, that's not about the risk exposure of some specific investment, it's about spreading our exposure to a variety of investment and income streams, so an undesired outcome in a single one doesn't take the whole pot. Mutual funds help us do that with respect to equity investment, but it's mostly up to the individual to manage that with other sources with more conservative risk-reward tendencies. FWIW, I've spread my income streams among two DB pensions, 401k, and SS. That way, a single bad event shouldn't take me out totally. Okay, asteroid strike...

In that regard, I'd look at Firecalc's MC option as a risk assessment tool. If the numbers out of it are significantly different from the historical tool, you might consider what it may be telling you about risk exposure...
__________________
ggbutcher is offline   Reply With Quote
Old 06-28-2014, 11:41 AM   #19
Thinks s/he gets paid by the post
 
Join Date: May 2014
Location: Utrecht
Posts: 2,204
Quote:
Originally Posted by Travelwanted View Post
Many simulations showed 95-100%. However, a couple showed 50%. In one run I had a chance to become a billionaire...seriously.

I realize this is telling me it is highly unlikely except under catastrophic conditions that we will not succeed. But I do not think it implies lower returns moving forward. I think it runs random and sometimes extreme variable and gives outputs resulting from henceforth unseen economic collapse.
Depending on your exact parameters, this is correct.

The issue you have is likely caused by two things:
  • Your simulation is memory-less. This means that the return of any given year does not depend on the previous year. This is a big difference with Firecalc.
  • The distribution you choose is not suited very well to model these things. Probably you have a normal distribution, or maybe even a uniform one?
Both combined mean that it is quite easy for the simulation to spit out a run where you have -50% returns for 10 years in a row, or conversely +100% for 10 years in a row.

This gives you the billion dollars and speedy bankruptcy outcomes.

Please keep in mind that a monte carlo simulation by definition runs random variables. It is up to you to decide how random you want it to be.
__________________
Totoro is offline   Reply With Quote
Old 06-28-2014, 12:39 PM   #20
Thinks s/he gets paid by the post
 
Join Date: Jun 2014
Posts: 1,035
MC sims can give ridiculous results. Start with a million, compound for 20 years then spend 35k annually, and you are broke in 2 years. Makes no sense but total randomness can create that scenario where the sp500 can be bought as a penny stock then recover.

With MC I exclude the top and bottom 10% of results and consider them fantasy. With one, attacking aliens are driving the market down, with the other, I am breathing gold plated oxygen to show off my wealth.


Sent from my iPhone using Early Retirement Forum
__________________

__________________
dallas27 is offline   Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Monte Carlo Simulations-now recognized as understating risk samclem FIRE and Money 46 11-07-2009 12:49 PM
Isn't FireCalc and Monte Carlo simulations based on higher gains? modhatter FIRE and Money 17 10-02-2006 04:32 AM
monte carlo Mysto FIRE and Money 5 04-18-2006 07:34 PM
Monte Carlo now can be used by brokers newellcr FIRE and Money 3 02-18-2005 09:41 AM
Online Monte Carlo calculator RoyUtah FIRE and Money 5 01-18-2004 02:41 AM

 

 
All times are GMT -6. The time now is 01:58 AM.
 
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2017, vBulletin Solutions, Inc.