Join Early Retirement Today
Reply
 
Thread Tools Search this Thread Display Modes
Monte Carlo or Historical in cFIRE and Firecalc?
Old 02-26-2015, 03:53 PM   #1
Recycles dryer sheets
 
Join Date: May 2014
Posts: 109
Monte Carlo or Historical in cFIRE and Firecalc?

When I run my plan in cFIRE using the Historicals, I get 90%+ success rate. But then I switch it to Monte Carlo and I get 67%.

When you guys talk about your % success rates, is that using historicals or monte carlo? Should I be using Monte Carlo all the time? I've been using Historicals.
__________________

__________________
bulbar is offline   Reply With Quote
Join the #1 Early Retirement and Financial Independence Forum Today - It's Totally Free!

Are you planning to be financially independent as early as possible so you can live life on your own terms? Discuss successful investing strategies, asset allocation models, tax strategies and other related topics in our online forum community. Our members range from young folks just starting their journey to financial independence, military retirees and even multimillionaires. No matter where you fit in you'll find that Early-Retirement.org is a great community to join. Best of all it's totally FREE!

You are currently viewing our boards as a guest so you have limited access to our community. Please take the time to register and you will gain a lot of great new features including; the ability to participate in discussions, network with our members, see fewer ads, upload photographs, create a retirement blog, send private messages and so much, much more!

Old 02-26-2015, 03:57 PM   #2
Give me a museum and I'll fill it. (Picasso) Give me a forum ...
REWahoo's Avatar
 
Join Date: Jun 2002
Location: Texas Hill Country
Posts: 42,129
Historical.
__________________

__________________
Numbers is hard

When I hit 70, it hit back

Retired in 2005 at age 58, no pension
REWahoo is offline   Reply With Quote
Old 02-26-2015, 04:02 PM   #3
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
 
Join Date: Sep 2005
Location: Northern IL
Posts: 18,288
The default for FIRECalc is historical, I'd assume anyone doing Monte Carlo would specify that. IMO, almost everyone using those tools with historical data is using the historical mode - it is what they are designed to do, and makes them different (and far more valuable IMO), than the others.

With Monte Carlo, you never really know what you are getting. How much variation did the programmer put into it, did they try to match historical correlation of different investments and inflation? If so, why not just use historical.

Nothing can predict the future, but I'd rather base some numbers on what we know has happened, rather than some programmer's model. And they don't usually share their code, so how can you check it? At least with historical, you can make some checks - the data is public.

PS - in case you didn't catch my drift, I don't care for and don't use Monte Carlo - unless I'm doing some engineering analysis, it has a place in some cases, but not this one, IMO.

-ERD50
__________________
ERD50 is offline   Reply With Quote
Old 02-26-2015, 04:02 PM   #4
Moderator
MichaelB's Avatar
 
Join Date: Jan 2008
Location: Rocky Inlets
Posts: 24,487
Historical for FIRECalc
__________________
MichaelB is offline   Reply With Quote
Old 02-26-2015, 04:06 PM   #5
Full time employment: Posting here.
 
Join Date: Feb 2014
Posts: 731
Quote:
Originally Posted by bulbar View Post
When I run my plan in cFIRE using the Historicals, I get 90%+ success rate. But then I switch it to Monte Carlo and I get 67%.

When you guys talk about your % success rates, is that using historicals or monte carlo? Should I be using Monte Carlo all the time? I've been using Historicals.
Both-just to get an idea of different results.

I also change some of the default values for return, std deviation, and inflation when running a MC scenario for "Portfolio with Random Performance" to bracket success rate.

Also, I've noticed that you get different MC success percentages running the exact same values due to the random number 'seed' that a given run may use. So, I make lots of MC runs to see what the spread of success may be.
__________________
BBQ-Nut is offline   Reply With Quote
Old 02-27-2015, 12:36 AM   #6
Thinks s/he gets paid by the post
Sunset's Avatar
 
Join Date: Jul 2014
Location: Chicago
Posts: 4,742
Oops, too late to do the Monte Carlo as we already retired
__________________
Sunset is offline   Reply With Quote
Old 02-27-2015, 12:58 AM   #7
Thinks s/he gets paid by the post
photoguy's Avatar
 
Join Date: Jun 2010
Posts: 2,301
MC can be useful IF you understand how the simulation works and the assumptions behind the model. It has the advantage of being able to run different scenarios (especially those that do not have good representation in the historical data)

But there are also a lot of drawbacks -- complex models are harder to understand, are not necessarily more accurate, easier to make implementation mistakes, and easier to fudge if one wants to push an agenda.


Sent from my iPad using Early Retirement Forum
__________________
photoguy is offline   Reply With Quote
Old 02-27-2015, 08:35 AM   #8
Recycles dryer sheets
robertf57's Avatar
 
Join Date: Jun 2014
Posts: 329
Given the stark differences, did you use a 40 year period of retirement or something? There are less 40 year periods in the historical database and they lead to a somewhat rosey projection because they under-represent some of the more challenging recent periods and over-represent some prosperous periods. I have used 30 year periods for my firecalc models...
__________________
robertf57 is offline   Reply With Quote
Old 02-27-2015, 09:40 AM   #9
Recycles dryer sheets
 
Join Date: Nov 2014
Posts: 381
By the way, Jim Otar's book has a good section on Monte Carlo. Worth a read, if you have a math background. Even if you don't, he recognizes that returns (at least the SP500) goes through long-ish periods of nearly constant growth at a certain growth rate (with noise around that, of course). He has a MC simulator that takes that into account, even though he doesn't like MC at all. Most MC simulations assume each year is completely random with no correlation year-to-year. Will the future do likewise - who knows?
__________________
big-papa is offline   Reply With Quote
Old 02-27-2015, 10:00 AM   #10
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
pb4uski's Avatar
 
Join Date: Nov 2010
Location: Vermont & Sarasota, FL
Posts: 16,455
The difference in your outcomes seems odd. I tested with firecalc assuming $1m portfolio and $40k withdrawals over 30 years, 60/40 portfolio and all other assumptions default assumptions.

Base case: historical: 95.6%
Monte Carlo w/8.8% average return, default std dev*: 98.6%, 100% and 95.9%
Vanguard Monte Carlo simulator: 93%, 92%, 93%

* historical average for 60/40 portfolio per Vanguard

I ran the Monte Carlo three times for each tool.
__________________
If something cannot endure laughter.... it cannot endure.
Patience is the art of concealing your impatience.
Slow and steady wins the race.
pb4uski is online now   Reply With Quote
Old 02-27-2015, 10:05 AM   #11
Full time employment: Posting here.
gcgang's Avatar
 
Join Date: Sep 2012
Posts: 928
I use a MC program, believing it's preferable to look at 2000+ sets of returns rather than one historical.




Sent from my iPad using Early Retirement Forum
__________________
In theory, there's no difference between theory and practice. In practice, there is. YB
gcgang is offline   Reply With Quote
Old 02-27-2015, 10:44 AM   #12
Recycles dryer sheets
 
Join Date: Jun 2014
Posts: 406
I use both but I'm not a big fan of FireCalc (or cFiresim)'s method of MC. I prefer Flexible Retirement planner for that since it is far more...flexible

Historical is nice but history may not repeat (the conditions that made those markets might not happen in the future I mean...like a couple of world wars and then booms after for example). The good and bad things of MC is that done correctly they will be totally random...so a decade or 2 of declines is possible (and the reverse). Not something that is necessarily likely but if you can survive that you are probably good to go. In FRP it shows the # of times the money ran out as well as the 10/90%ile levels so you can tell if the money ran out under a severe outlier condition or not.

I do get much better results from historical than from MC especially if I set up a scenario with a sharp decrease of say 30-50% (sort of comparing 1929 followed by MC rather than 1929 followed by the historical results).
__________________
If money is the root of all evil I want to be a bad man
nuke_diver is offline   Reply With Quote
Old 02-27-2015, 11:18 AM   #13
Recycles dryer sheets
 
Join Date: May 2014
Posts: 109
I tried Flexible Planner. It gives me 100% using a fixed 5.5% return, no st deviation. Then I used the monte carlo. I used the moderate risk assumptions and it gave me 85%. I tried the other risk categories and got about the same. Much higher than the 67% monte carlo I was getting with cFIRE.
__________________
bulbar is offline   Reply With Quote
Old 02-27-2015, 11:34 AM   #14
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
pb4uski's Avatar
 
Join Date: Nov 2010
Location: Vermont & Sarasota, FL
Posts: 16,455
Wouldn't 5.5% fixed with no standard deviation be the same as 5.5% deterministic?
__________________
If something cannot endure laughter.... it cannot endure.
Patience is the art of concealing your impatience.
Slow and steady wins the race.
pb4uski is online now   Reply With Quote
Old 02-27-2015, 12:48 PM   #15
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
 
Join Date: May 2004
Posts: 11,615
Quote:
Originally Posted by gcgang View Post
I use a MC program, believing it's preferable to look at 2000+ sets of returns rather than one historical.
2000 sets of notional returns or one set of real returns, right?

There's no perfect answer, and this has been studied a lot. There is definite correlation between years and between data sets. Some MC sims attempt to model these, some just go with random returns that fit into the defined SD window, but don't attempt to match the corelations of real life. To use data that assumes (for example) that inflation is not related to the returns on fixed income investments seems to be not very useful. And if the MC has been carefully designed to match real-world correlations--at some point it makes just as much sense to use the real world data raw.
Some models I've seen use real data from more than one country to help retain the correlations while allowing for a different set of meta-economics (i.e. not just/primarily 20th century US).
__________________
"Freedom begins when you tell Mrs. Grundy to go fly a kite." - R. Heinlein
samclem is online now   Reply With Quote
Old 02-27-2015, 11:30 PM   #16
Recycles dryer sheets
 
Join Date: May 2013
Posts: 119
Quote:
Originally Posted by samclem View Post
2000 sets of notional returns or one set of real returns, right?

There's no perfect answer, and this has been studied a lot. There is definite correlation between years and between data sets. Some MC sims attempt to model these, some just go with random returns that fit into the defined SD window, but don't attempt to match the corelations of real life. To use data that assumes (for example) that inflation is not related to the returns on fixed income investments seems to be not very useful. And if the MC has been carefully designed to match real-world correlations--at some point it makes just as much sense to use the real world data raw.
Some models I've seen use real data from more than one country to help retain the correlations while allowing for a different set of meta-economics (i.e. not just/primarily 20th century US).
An endless source of debate here! There are a number of different ways of doing MC, and, IMHO, one of the best ways is with the bootstrap (discussed previously here); you don't have to assume a distribution. Although fixed income time series do show autocorrelation, this isn't true for annual stock returns. And there are methods (for instance, block bootstrap), that deal with this problem.

I'm not a big fan of non-MC Firecalc, and this is confirmed (at least for me) by the fact that MC tends to be more pessimistic. I'm also uncomfortable with the small number (about 4 or 5) of truly independent time series that FC uses to predict outcomes.
__________________
Fred123 is offline   Reply With Quote
Old 02-28-2015, 11:15 AM   #17
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
 
Join Date: Sep 2005
Location: Northern IL
Posts: 18,288
Quote:
Originally Posted by Fred123 View Post
...

I'm not a big fan of non-MC Firecalc, and this is confirmed (at least for me) by the fact that MC tends to be more pessimistic. ...
And a few minor tweaks of the programmer's MC terms and/or algorithms will change that. So when is it 'right'?

What if M-C was less pessimistic than history? Would it be 'wrong'? Sounds like you are still using history as a base-line. Why not use history, plus a comfortable safety factor? I think MC is just a fancy, convoluted, complicated, circular means to that end. 'But a computer came up with this answer!'



Quote:
I'm also uncomfortable with the small number (about 4 or 5) of truly independent time series that FC uses to predict outcomes.
Agreed, but since we can't predict the future, I thinks it's all we got. At some point, most of us just need to go with our decision, or work until we die.

I feel reasonable, going with 100% and ~ 45 years (before you start dropping off bad cycles), knowing that my portfolio should survive the worst of anything we've seen in history, plus a little fudge factor for safety. Just because we may have some future cycles that are unrepresented in history does not mean they will be worse than the worst ever thrown at us. And if they do, I've got some buffer. and will need to adjust - probably like everyone else, regardless what tool they used to determine their CWR (Comfortable Withdraw Rate).

-ERD50
__________________
ERD50 is offline   Reply With Quote
Old 02-28-2015, 05:33 PM   #18
Recycles dryer sheets
 
Join Date: May 2013
Posts: 119
Quote:
Originally Posted by ERD50 View Post
And a few minor tweaks of the programmer's MC terms and/or algorithms will change that. So when is it 'right'?

What if M-C was less pessimistic than history? Would it be 'wrong'? Sounds like you are still using history as a base-line. Why not use history, plus a comfortable safety factor? I think MC is just a fancy, convoluted, complicated, circular means to that end. 'But a computer came up with this answer!'





Agreed, but since we can't predict the future, I thinks it's all we got. At some point, most of us just need to go with our decision, or work until we die.

I feel reasonable, going with 100% and ~ 45 years (before you start dropping off bad cycles), knowing that my portfolio should survive the worst of anything we've seen in history, plus a little fudge factor for safety. Just because we may have some future cycles that are unrepresented in history does not mean they will be worse than the worst ever thrown at us. And if they do, I've got some buffer. and will need to adjust - probably like everyone else, regardless what tool they used to determine their CWR (Comfortable Withdraw Rate).

-ERD50
So, how exactly do you choose that fudge factor? The problem is that if you have no estimate of the variance of the distribution, then it's impossible to make any kind of intelligent guess as to what might work. The beauty of MC is that it provides a method for estimating the variance (or randomness if you like) in the distribution and gives you an idea of what that fudge factor should be!

And, really, there's nothing at all complicated about MC. In fact, it couldn't be simple. In the case of bootstrapping, if you're interested in a 30 year retirement period, you just select 30 random years from your historical financial time series and calculate the outcome in the same way as FC. Repeat this a 1000 times and you have your estimate.
__________________
Fred123 is offline   Reply With Quote
Old 02-28-2015, 06:21 PM   #19
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
 
Join Date: Sep 2005
Location: Northern IL
Posts: 18,288
RE: fudge factor on historical...

Quote:
Originally Posted by Fred123 View Post
So, how exactly do you choose that fudge factor? The problem is that if you have no estimate of the variance of the distribution, then it's impossible to make any kind of intelligent guess as to what might work. The beauty of MC is that it provides a method for estimating the variance (or randomness if you like) in the distribution and gives you an idea of what that fudge factor should be! ...
That would be easier for me to consider if there was some absolute definition for the M-C factors for an application such as this. But, AFAIK, it is up to the programmer to determine whether they are going to make any adjustments for correlations between different inputs (inflation versus interest rates, etc).

Quote:
And, really, there's nothing at all complicated about MC. In fact, it couldn't be simple. In the case of bootstrapping, if you're interested in a 30 year retirement period, you just select 30 random years from your historical financial time series and calculate the outcome in the same way as FC. Repeat this a 1000 times and you have your estimate.
And there is the crux of the problem - I don't believe that financial markets are that simple. Yes, there is randomness year-to-year, but booms follow busts, and busts follow booms, with some flat levels in between, sometimes. I think there are some complex reasons for that, including human emotion and the 'madness of crowds'. The inflection points change, and the time and flat periods vary (they do this just to foul up any dirty-market-timer attempts! .

-ERD50
__________________
ERD50 is offline   Reply With Quote
Old 02-28-2015, 06:22 PM   #20
Recycles dryer sheets
galeno's Avatar
 
Join Date: Nov 2002
Location: Alajuela, Costa Rica
Posts: 220
For our 60/40 port with a TER = 0.9%.


Historical Firecalc gives us an AWR = 3.6% for an 85% success rate.


Monte Carlo ORP gives us a 3.5% AWR.
__________________

__________________
AA = 60/35/5. Expected CAGR = 5.7%. GSD (5y) = 7.8%. USD inflation (10 y) = 1.8%. AWR = 3.0%. TER = 0.5%. Net Port Yield = 1.7%. Term = 36 yr. FI Duration = 4.9 yr. Portfolio survival probability = 86%.
galeno is offline   Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
cFIRE question about monte carlo versus historicals bulbar FIRE and Money 19 01-15-2015 04:46 AM
monte carlo vs. firecalc - or what is fido doing to me! Mysto FIRE and Money 17 03-26-2007 03:00 PM
Question on Firecalc Monte Carlo macnjus FIRECalc support 15 11-24-2006 12:17 PM
FireCalc = Monte Carlo simulator DW FIRECalc support 1 11-19-2006 07:46 PM
Isn't FireCalc and Monte Carlo simulations based on higher gains? modhatter FIRE and Money 17 10-02-2006 04:32 AM

 

 
All times are GMT -6. The time now is 04:50 PM.
 
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2017, vBulletin Solutions, Inc.