FireCalc Weakness?

[Senator]

I have been delving into many different retirement planners. My first iterations were to just plug in numbers, and hit ‘play’. All seems to be well. Now I am looking under the covers at many of them, to see what defaults are set, and how they actually work. All have their strengths and weaknesses.

I like FireCalc as it uses actual market performance. Others just use some sort of statistical value of what they consider ‘average’. Some use a standard deviation, some use Monte Carlo simulations.

If I use FireCalc, for 30 years, it starts using actual market conditions, beginning with 1871 and goes forward for 30 years. Then it tries again in 1872 and goes forward 30 years. Since I need 30 years, it cannot use market projections after 1984. If I use 40 years, it misses market returns after 1974.

That means the dot.com bubble in 2000 is missed. The market crash of 1987 is missed. The great recession of 2007 is missed, etc. It does hit the 1929 depression and the early 70s and 80s stock market crashes, etc.

Maybe the 114 (or 104) years are typical enough to project, but it is something I noticed. If the last 30 or 40 years are quite a bit different than the years between 1871 and 1985, it could be quite a different FIRE plan.

Any thoughts? Does it make any statistical difference if 1985 to current is excluded?

 

[ProspectiveBum]

Technically, the periods after 1985 aren't excluded, they just won't be used as the beginning of your investment period, which makes sense. So one of the 30 year windows will include a period starting in 1984, running through 2014 (so including the internet bubble and more recent meltdown, as well as the recent bull market returns).

Other 30 year windows will include the crash of '29, stagflation of the 70's, the market crash in the late 80's, etc.

You're right that there's a risk that if an event like the recent market crash happened immediately after retirement, we don't have long-term data about how that'd affect ones portfolio. Given that the market has rebounded to new highs within a few years, I'd guess that anybody who had a sensible AA will be fine.


Sent from my iPhone using Early Retirement Forum

 

[walkinwood]

If I use FireCalc, for 30 years, it starts using actual market conditions, beginning with 1871 and goes forward for 30 years. Then it tries again in 1872 and goes forward 30 years. Since I need 30 years, it cannot use market projections after 1984. If I use 40 years, it misses market returns after 1974.

That means the dot.com bubble in 2000 is missed. The market crash of 1987 is missed. The great recession of 2007 is missed, etc. It does hit the 1929 depression and the early 70s and 80s stock market crashes, etc.

Not entirely correct. There are no 30/40 year periods that start after 1984/1974, but the data after 1984/1974 is used in periods that start before those years. Data from 2013 is used in only one run in each case, data from 2012 in two runs, etc.

Definitely a shortcoming, but something that is built into the methodology. You could use shorter retirement periods to see what happens in those time frames. Starting in year 2000 and going for 13 years is quite informative.

Statistically, there are some who question the value of using overlapping periods, others who say that the number of iterations is too small (~100) to make an informed decision. (I don't have an opinion on the statistical validity since I don't understand it at that level - but have based my retirement on historical calculations)

 

[Senator]

There are no 30/40 year periods that start after 1984/1974, but the data after 1984/1974 is used in periods that start before those years.

Good point, both yourself and ProspectiveBum. That makes sense, and hopefully anything that would start after 1985 would be duplicated, or worse, by a 1929 start date.

I am always trying to find a flaw in any calculations as I get close. It seems as though I am trying to build a house in Hawaii that can withstand sub-zero tempertaures...

 

[CaliforniaMan]

We have the history we have, so that is not a FIRECalc flaw, just what we have. The primary flaw in FIRECalc, as I understand it, is that it does not update the value of bond holdings as interest rates change. Depending on the duration of your bond portfolio, the value will of course increase or decrease with decreasing or increasing interest rates and this change is not used (as I understand it) in the FIRECalc rebalancing or portfolio values and survival results.

 

[David1961]

I know there are posters here who ER'd at market highs at either 2000 or 2007 and they seem to be doing fine. I think as long as you are flexible and can adjust your spending (if necessary), you will be fine. IMO, no planner can predict the future with 100% accuracy.

 

[MichaelB]

I know there are posters here who ER'd at market highs at either 2000 or 2007 and they seem to be doing fine. I think as long as you are flexible and can adjust your spending (if necessary), you will be fine. IMO, no planner can predict the future with 100% accuracy.

That's us. Still hanging in there, as you can see by my sig line. No major changes or cutbacks.

 

[Travelwanted]

I am always trying to find a flaw in any calculations as I get close. It seems as though I am trying to build a house in Hawaii that can withstand sub-zero tempertaures...

Lol...so true. I think you referred to this same OVER analyze issue in another post. I am guilty as well!

 

[nuke_diver]

Guilty as well

I like that you can use historical data and I cross those results with Monte Carlo results. While the historical starts will be limited to your time frame the MC results can also be unrealistic...20 down years in a row is actually possible (saw it happen once when I ran out of money really early and wondered why). THe Flexible Retirement Planner does MC simulations but throws out the top and bottom 10% to try and avoid something like that

 

[retirementguy1]

I think I remember William Bernstein or Bob Clyatt saying that anything over 80% on Firecalc (and probably any retirement calculator) is meaningless. Choose an AA that works for you and rebalance when necessary.

 

[razztazz]

I know there are posters here who ER'd at market highs at either 2000 or 2007 and they seem to be doing fine. I think as long as you are flexible and can adjust your spending (if necessary), you will be fine. IMO, no planner can predict the future with 100% accuracy.

I member here who hasn't posted in a while but apparently has his own website did a break out of a typical Y2K retiree effective Dec 2012. (standard 60/40 asset allocation) It was looking pretty gloomy. Gonna need a long run of really good returns for the retiree to survive. I don't know what this would look like as of today.

I can post the link to his figures if you want.

 

[razztazz]

I think I remember William Bernstein or Bob Clyatt saying that anything over 80% on Firecalc (and probably any retirement calculator) is meaningless. Choose an AA that works for you and rebalance when necessary.

Bernstein was referring to using Monte Carlo systems. Using historical data of every kind and cause of societal collapse, he, apparently, concluded we all have only an 80% chance of living a normal lifespan.

 

[ejman]

I think the primary fault in the interpretation moving forward of the results that Firecalc presents is that the data set reflects the time period the US became the preeminent world power. I've seen results from studies of WR rates for other countries and the US, along with Canada is pretty much at the sweet end of the spectrum. Other industrialized nations have fared much worse with maximum WR in the 1 to 3% range.

Of course, nobody knows what the future holds and the next 150 years might be even sweeter...

 

[ejman]

I think I remember William Bernstein or Bob Clyatt saying that anything over 80% on Firecalc (and probably any retirement calculator) is meaningless. Choose an AA that works for you and rebalance when necessary.

Here is the Retirement calculator from hell article I think you were referring to The Retirement Calculator from Hell, Part III

 

[RobLJ]

I noticed increasing the time in FireCalc from 30 to 35 and then to 40 years actually increased the success rate from 92% to 100% in the last two scenarios, which didn't make sense. Until I read this post. Decreasing to 25 or 20 years might actually make it go down further (no--the max was 3 scenarios failing). This ignores reducing spending.

 

[Telly]

Many years ago on this forum I pointed out the effect in FireCalc that as the time period is increased, the number of data years excluded as start dates increases right with it. For example, today a 30-year period can not start with 1986, etc.

At the time, I made the suggestion to include non-complete periods in the analysis, so they would show up in the graphing function (the myriad of paths) so a user could see them. After some discussion, Dory added that function. In a later FireCalc update, that feature was dropped and duly noted in the change notes for the upgrade that it was dropped.

I tried to look back over posts I started, and have not been able to find that original discussion. Maybe I did a reply on a topic.

Back then it was important to me, as I was running FireCalc with long periods. Today for me it has decreased in importance with my perceived decrease in longevity due to the passage of time

Come to think of it, I don't think I have run FireCalc in years! The last time I ran it, I had to select "Classic" FireCalc to get the original-style calculator.

 

[GTFan]

I member here who hasn't posted in a while but apparently has his own website did a break out of a typical Y2K retiree effective Dec 2012. (standard 60/40 asset allocation) It was looking pretty gloomy. Gonna need a long run of really good returns for the retiree to survive.

But again, these calcs all assume that the retiree doesn't adjust any WRs, reduce expenses etc. in the face of poor returns soon after retirement. This is highly unrealistic IMO and is one of the basic flaws using this data from any worst-case run.

 

[Looking4Ward]

But again, these calcs all assume that the retiree doesn't adjust any WRs, reduce expenses etc. in the face of poor returns soon after retirement. This is highly unrealistic IMO and is one of the basic flaws using this data from any worst-case run.

+1

Agreed. I just did a quick run using a 5% of portfolio balance withdrawal calculation for a 14 year period beginning with the year 2000 and although I wouldn't have ended with as much as I started, I would have been able to live comfortably on all of the withdrawal amounts and still have a good portion of my original portfolio remaining.

 

[razztazz]

This is highly unrealistic IMO and is one of the basic flaws of any worst-case run.

For the most part I agree. Some folks who were forced to retire due to medical or other force majeure and had to just go with the 4% and had little wiggle room have their hands tied on that and might have some white knuckles right now.

However, eventually they'll get Soc Sec and that should plug that gap before they run out of money or reach the true point of no return.

Based on my own spending I'd be better off now if I had gotten a lump sum instead of the pension, so yes, if one had the financial flexibility one way or another you're OK

 

[GTFan]

Agreed. I just did a quick run using a 5% of portfolio balance withdrawal calculation for a 14 year period beginning with the year 2000 and although I wouldn't have ended with as much as I started, I would have been able to live comfortably on all of the withdrawal amounts and still have a good portion of my original portfolio remaining.

Which calculator did you use to look at that? Would be curious to see how mine would do over that time and with my intended WR, given that it's a good worst case.

 

[Midpack]

I think I remember William Bernstein or Bob Clyatt saying that anything over 80% on Firecalc (and probably any retirement calculator) is meaningless. Choose an AA that works for you and rebalance when necessary.

Many people continue to misquote him, unfortunately that's well outside the context of Dr B's statement from the Retirement Calculator From Hell series. He pointed out that based on the long arc of history, a larger geopolitical or some other unpredictable event may override any market circumstances we've seen (which already includes some substantial geopolitical events) - particularly in the USA.

Barring some historical catastrophe, a probability of 160% (ie, 2% WR) is still statistically twice as safe as 80% (ie,4% WR). A comet might hit your house, throwing any probability of success out the window - but planning for the comet to miss has some merit...

http://www.efficientfrontier.com/ef/901/hell3.htm

 

[FIRE'd@51]

The primary flaw in FIRECalc, as I understand it, is that it does not update the value of bond holdings as interest rates change. Depending on the duration of your bond portfolio, the value will of course increase or decrease with decreasing or increasing interest rates and this change is not used (as I understand it) in the FIRECalc rebalancing or portfolio values and survival results.

I don't think this is correct. FireCalc uses total returns for each asset class, so it would implicitly take into account urnealized capital gains and lossses on bonds. What it doesn't do is break-out the income and price-change for each asset class.

 

[haha]

I think I remember William Bernstein or Bob Clyatt saying that anything over 80% on Firecalc (and probably any retirement calculator) is meaningless. Choose an AA that works for you and rebalance when necessary.

But how do you determine what AA "works for you".

I thought that was what the calculators were for?

Ha

 

[Longview]

Re- longer time horizons in Firecalc. If you want a really real worst case scenario, say for a 40 year horizon, put in a 20 year horizon -- then take the lowest from that and put it into another 20 year horizon. (Or use the average if you don't want to be super negative)

The idea is a shorter time horizon gives you more "runs" -- ie, if you put in a 60 year time horizon then you are culling out the randomized last 59 years from your data set.

 

[gcgang]

Bernstein was referring to using Monte Carlo systems. Using historical data of every kind and cause of societal collapse, he, apparently, concluded we all have only an 80% chance of living a normal lifespan.

As all retirement planner calculators are dealing with uncertainty of future returns, your TARGET success rate with the 2000+ simulations run on a good Monte Carlo simulation SHOULD be in the 75-90% range. Don't concern yourself with the top or bottom 10%, just re-run periodically to adjust your AA and spending to stay 75-90%.

If you are getting close to 100%, you are in effect leaving a much larger legacy than you intended in your inputs, and are taking more risk than you need.


Sent from my iPad using Early Retirement Forum