Using Monte Carlo Analysis in FireCalc

[nico08]

Hi. I have a question. I am using monte carlo analysis option in FireCalc. I keep the variables the same and have just done multiple runs. These various runs have produced results anywhere between a 69% likelihood of early retirement success to a 100% likelihood of early retirement success. I don't understand why I am getting different results on these runs when I am making no other changes to the variables.

How do I assess my likelihood of early retirement success using the monte carlo analysis option in FireCalc if I get such a wide range of results?

Thank you.

 

[ERD50]

Hi. I have a question. I am using monte carlo analysis option in FireCalc.

I have a question. Why (use Monte Carlo)?

I keep the variables the same and have just done multiple runs. These various runs have produced results anywhere between a 69% likelihood of early retirement success to a 100% likelihood of early retirement success. I don't understand why I am getting different results on these runs when I am making no other changes to the variables.

It is because you are using Monte Carlo. It runs a randomized sequence, with bounds and algorithms known only to the programmer (maybe not even to the programmer). Each time you run it, it will use a new 'seed' to feed the pseudo-random routine to generate a different set of sequences.

How do I assess my likelihood of early retirement success using the monte carlo analysis option in FireCalc if I get such a wide range of results?

My suggestion (like the old patient/doctor joke), don't do that!

Thank you.

You're welcome.

PS - stick to the historical calculators.

-ERD50

 

[nico08]

Hi Erd50.

Thanks for your response. I definitely rely more on historical calculators over monte carlo calculators. I just can't understand the use/purpose of a monte carlo analysis if it will produce such a broad range of results. How is that useful in any way, if you are trying to determine the likelihood of success in your early retirement plan?

 

[ERD50]

Hi Erd50.

Thanks for your response. I definitely rely more on historical calculators over monte carlo calculators. I just can't understand the use/purpose of a monte carlo analysis if it will produce such a broad range of results. How is that useful in any way, if you are trying to determine the likelihood of success in your early retirement plan?

That's my feeling as well.

There are others who feel that by doing Monte Carlo, you are simulating things that are worse/different than the historical past. OK, but I just don't see how you determine what is realistic or not? So I don't use them.

Some even get circular, and try to demonstrate how close the Monte Carlo is to historical. OK, why not use historical, which can be independently tested and verified and understood?

Monte Carlo is included in some historical calculators, I think just because people expect them to have it. But I just don't feel they are useful.

My personal comfort level is to strive for 100% historical success, and throw in a buffer. At that point, if I fail, I can at least say that I had a plan that should survive even the worst times with a little room to spare.

-ERD50

 

[RetireAge50]

With a coin flip model (Monte Carlo) you can run it 500 times and get different results. However, this is a good thing as you will get a distribution of outcomes which should give you a feel of the probabilities. The historical calculators do a similar thing using actual sequences instead of random.

 

[photoguy]

Hi. I have a question. I am using monte carlo analysis option in FireCalc. I keep the variables the same and have just done multiple runs. These various runs have produced results anywhere between a 69% likelihood of early retirement success to a 100% likelihood of early retirement success. I don't understand why I am getting different results on these runs when I am making no other changes to the variables.

How do I assess my likelihood of early retirement success using the monte carlo analysis option in FireCalc if I get such a wide range of results?

Thank you.

The MC run is probably not setup correctly in firecalc. With MC, either you or the program should set the number of trials so that the outcome is stable. If it is not stable, chances are it's set too low.

 

[nico08]

hi. do you know if you can set the number of monte carlo trials in firecalc?

 

[REWahoo]

hi. do you know if you can set the number of monte carlo trials in firecalc?

See this 2006 post by the creator of FIRECalc describing the Monte Carlo process: http://www.early-retirement.org/forums/f36/question-on-firecalc-monte-carlo-24202.html#post449961

 

[papadad111]

With a coin flip model (Monte Carlo) you can run it 500 times and get different results. However, this is a good thing as you will get a distribution of outcomes which should give you a feel of the probabilities. The historical calculators do a similar thing using actual sequences instead of random.

+1. It's the distribution plot of thousands of random possible outcomes that in turn help visualize / understand the probabilities - that data makes a Monte Carlo analysis insightful. The plot of the data points in a distribution graph helps to interpret the Monte Carlo data.

 

[Travelwanted]

I loved monte carlo when it told me I would have 500 million in 30 years , but hated it when it said I had a 60% chance of success using the exact same numbers.

For my blood mc is just too random. Wait, that is redundant.

 

[nuke_diver]

I don't dismiss MC sim out of hand, they are simply another tool. I don't like Firecalc version because I'm not that sure of what it is doing. The best MC that I have used throws away the top and bottom 10% results. This avoids the crazy big or small numbers.

Since we do not know the future and cannot be sure of scenarios that have never happened before running both historical and MC broadens the range of conditions that could occur. The one thing that MC don't do (and can't) is the psychology of market and I could argue that historical doesn't either. The crashes occurred in different times with different economic conditions to those of today or of 30 years from now. So the future crashes (which is really what we worry about the most) will be triggered by events that have not happened before more than likely. As a speculative example, a solar flare powerful knocking out communications (but not so powerful to fire the electric grid) for days or weeks would be a major disaster today (no cell phone/internet) but in 1929 might have been a minor inconvenience if even noticed

 

[ERD50]

I don't dismiss MC sim out of hand, they are simply another tool. .... The best MC that I have used throws away the top and bottom 10% results. This avoids the crazy big or small numbers. ...

This is exactly what seems so odd to me. MC users like MC because it gives results outside of historical limits. But then they decide to throw out some of the results as being 'too far' out. It just seems so arbitrary to me.

We can't predict the future. If you are the conservative type, you throw in some buffer. How much? I look at history, and throw in best-guess. The MC user runs the MC simulation, and then throws in their best guess and over-rides the results by throwing out some of the data, or tweaking the correlations or deviations or whatnot.

I fail to see how the middle step of running the MC adds any value. I guess I'm just an Occam's Razor kind of guy.

I don't like Firecalc version because I'm not that sure of what it is doing.

It seems to me that the historical reporting calculators are far more transparent than any MC I've see. The data is available, and you can test it with different runs and get what should be predictable changes to the output, based on input changes. The random nature of MC, and the fact that most programmers don't publish the algorithms, puts MC much more in the land of being 'not that sure of what it is doing'.

-ERD50

 

[Options]

IIRC, OTAR dislikes MC, VG finds it valid and useful, as does Pfau. My problem with using only historical is it's U.S. bias, covering a particularly positive time in American history. While Buffett may be betting on a positive American future, I'm not him and don't have his financial cushion. I prefer using both historical and MC, and even then add a cushion. Others may not be as conservative.

 

[papadad111]

My problem with using only historical is that it's USA bias, covering a particularly positive era in American financial history .

It's interesting how many current retirement tools (not just firecalc) are so reliant on historical USA equities (and inflation) data to perform future predictions via back-testing, yet at the same time most people's portfolio hold a significant position in non-USA equities.

Are we thus lulled into a false sense of comfort based solely on US equity financial history ?

I am reminded that the first thing the disclaimers say is that history is no indication of future performance.

What would firecalc say if we included as historical data the Japanese historic stock market returns, or China stock market returns or even Germany (or EAFE MSCI ) historical returns as a mix/ percentage of the said history ?

 

[nuke_diver]

It seems to me that the historical reporting calculators are far more transparent than any MC I've see

I mean the Firecalc MC not the historic of course that is straightforward. Throwing out the highs and lows makes perfect sense to me (manufacturing engineer) since these will be outlier results and not representative of the most of the distribution from the calculation. Firecalcs MC doesn't (I think) throw away any values so impossible results are possible

I believe in getting as much data as possible to form an opinion, seeing the results before rejecting them rather than just rejecting them out of hand without seeing. That's me, everyone is different. You prefer only historical, I like both since as others say the US market and market performance is not even the only market performance that could be (maybe even should be) considered. Since it is all a guess there is no reason not to believe that the US market could follow the performance of the JPN market over the last few decades, just because it never has doesn't mean it cannot.

But somehow I think we will end up agreeing to disagree

 

[ERD50]

I mean the Firecalc MC not the historic of course that is straightforward.

Ahh, I see now - sorry, I misunderstood.

As to the rest, sure, it is US-centric. I'm not sure how a retiree would fare with a Japan-like economy, but at least from what I've seen, the pictures of Japan (other than during the Tsunami) don't look like the pictures we see of the Great Depression. There might be some mitigating effects?

I bet someone has done the math on that though, maybe I'll do some searching later.

But somehow I think we will end up agreeing to disagree

I think we agree more than we disagree. We both want to add a buffer for the unknown. The way I see it, I just add the buffer that I am comfortable with, you run a tool until it produces a buffer that you are comfortable with (throwing 'outliers' - but are they?). Other then the middle step of running a tool, are we different?

But I think I made that point already, so I'll try to bow out until/if something new comes along.

-ERD50

 

[photoguy]

I've had some disagreements with ERD50 on the value of MC methods but I think this point he makes is very important. I wouldn't seriously consider outputs from any MC method without a clear idea of how the underlying algorithm works (and/or having the worked vetted through replication by others).

It seems to me that the historical reporting calculators are far more transparent than any MC I've see. The data is available, and you can test it with different runs and get what should be predictable changes to the output, based on input changes. The random nature of MC, and the fact that most programmers don't publish the algorithms, puts MC much more in the land of being 'not that sure of what it is doing'.