

SWR's and Success Probabilities
12072013, 06:00 AM

#1

Thinks s/he gets paid by the post
Join Date: Aug 2006
Posts: 2,241

SWR's and Success Probabilities
We all know that the rolling 30year periods used in FireCalc are not statistically independent due to data overlap.
I'm not a statistician, but I've become increasingly concerned that the SWR success rates from FireCalc may be significantly overestimated, due to this overlap which introduces significant serial correlation into the data series. I do know that serial correlation introduces a downward bias in a standard deviation calculation, and it's standard deviation that leads to the success probabilities. This may partially explain why Monte Carlo simulations, which use a standard deviation calculated from nonoverlapping data intervals (e.g. annual returns) tend to give lower SWR's than FireCalc. IOW, I'm suggesting that a 4% SWR could actually lead to a significantly lower success rate than 95% due to this data overlap effect; and, since we really don't know what the probability of success is, one should rely upon the FireCalc SWR that has never failed.
__________________
__________________
I'd rather be governed by the first one hundred names in the telephone book than the Harvard faculty  William F. Buckley




Join the #1 Early Retirement and Financial Independence Forum Today  It's Totally Free!
Are you planning to be financially independent as early as possible so you can live life on your own terms? Discuss successful investing strategies, asset allocation models, tax strategies and other related topics in our online forum community. Our members range from young folks just starting their journey to financial independence, military retirees and even multimillionaires. No matter where you fit in you'll find that EarlyRetirement.org is a great community to join. Best of all it's totally FREE!
You are currently viewing our boards as a guest so you have limited access to our community. Please take the time to register and you will gain a lot of great new features including; the ability to participate in discussions, network with our members, see fewer ads, upload photographs, create a retirement blog, send private messages and so much, much more!

12072013, 07:24 AM

#2

Give me a museum and I'll fill it. (Picasso) Give me a forum ...
Join Date: Feb 2006
Posts: 8,105

I guess I don't understand why the math about correlation would would matter. Firecalc is used to discover how you would have faired in any X period in financially recorded history. The answer is what it is. Whether you assume that the overlap you are concerned with is likely a feature of the world and thus likely to be repeated in the future is another matter.
__________________
__________________
Every man is, or hopes to be, an Idler.  Samuel Johnson



12072013, 08:38 AM

#3

Give me a museum and I'll fill it. (Picasso) Give me a forum ...
Join Date: Jan 2008
Location: Chicagoland
Posts: 10,663

FIRECALC doesn't "overestimate" anything, it's based on actual history. I wouldn't consider Monte Carlo superior by any means, it's another worthwhile method to consider that's all, but YMMV.
Quote:
Originally Posted by FIRECALC
How can FIRECalc predict future returns from past performance?
It can't. And it doesn't try. In fact, it tries to predict what will not happen. This might sound confusing, but it's really simple.
Consider an analogy: Suppose you are building a house in Honolulu. How do you decide how much heating and air conditioning capacity you will need? We know that the lowest it has ever been there was 52°, on a day in February 1902 and again on another day in January 1969, and the hottest it has ever been was 95°, in 1994. Buying a system suitable for an Anchoragestyle winter and a Phoenixstyle summer would be a major waste of money that could be better used elsewhere.
FIRECalc works the same way, using stock market history and your portfolio and spending plan instead of weather history and furnace capacity. No one could predict the temperature for any specific given future date during the decades that house will be used, and no one can predict the future returns of your investments. But by knowing the historical worst cases, you'll have the information to judge if your savings are sufficient to handle the winter.

__________________
No one agrees with other people's opinions; they merely agree with their own opinions  expressed by somebody else. Sydney Tremayne
Retired Jun 2011 at age 57
Target AA: 60% equity funds / 35% bond funds / 5% cash
Target WR: Approx 2.5% Approx 20% SI (secure income, SS only)



12072013, 08:57 AM

#4

Give me a museum and I'll fill it. (Picasso) Give me a forum ...
Join Date: Sep 2005
Location: Northern IL
Posts: 16,437

Quote:
Originally Posted by donheff
I guess I don't understand why the math about correlation would would matter. Firecalc is used to discover how you would have faired in any X period in financially recorded history. The answer is what it is. Whether you assume that the overlap you are concerned with is likely a feature of the world and thus likely to be repeated in the future is another matter.

Agree with previous posts. It is what it is. It reports what would have happened if you retired in 1966, or in 1965, etc. So 29 of those years overlapped? They did in real life as well.
And the way most of use FIRECalc (I assume), is to focus on the failures. So most of the data isn't even taken into consideration. Sure, it's nice to know that some X% of the time we would have ended up with a high multiple of our initial buying power, or that a significant % of the time we would have maintained our buying power, but the failures are the main concern. And they are what they are.
ERD50
__________________



12072013, 08:59 AM

#5

Thinks s/he gets paid by the post
Join Date: Jun 2010
Posts: 2,229

As others have mentioned FIRECALC reports on the exact number of runs in HISTORY that would have failed.
It doesn't produce what the probability of success is going forward (which is THE NUMBER that everyone wants). If you want to use the FIRECALC number as a forward estimate you should probably be concerned with two types of errors:
Bias which would be if the historical Firecalc success rate systematically under or overestimates the probability going forward. I think many people believe that Firecalc is biased to produce higher success rates (as a future estimate) because currently PE10 is high right now and bond yields are low. This is why so many people take a haircut on the allowable withdrawal rate (e.g., instead of 4% I'll use 3.5%).
Variance which would be due to fluctuations in the observed rate due to chance (but not systematic errors). For example, if you flipped a coin 10 times you might get 4 heads and estimate the probability of getting heads as 40%. In this case, due to variance the estimate is 10% too low.
Firecalc does not produce any sort of confidence bounds. To deal with this, you could mentally add +/ 10% to whatever number firecalc issues (I guess someone could calculate the impact of the overlapping periods by running simulations to figure out a better estimate than this).
__________________



12072013, 09:11 AM

#6

Thinks s/he gets paid by the post
Join Date: May 2006
Location: west coast, hi there!
Posts: 4,827

Quote:
Originally Posted by FIRE'd@51
...This may partially explain why Monte Carlo simulations, which use a standard deviation calculated from nonoverlapping data intervals (e.g. annual returns) tend to give lower SWR's than FireCalc. IOW, I'm suggesting that a 4% SWR could actually lead to a significantly lower success rate than 95% due to this data overlap effect; and, since we really don't know what the probability of success is, one should rely upon the FireCalc SWR that has never failed.

I don't know what is built into Monte Carlo simulators. Clearly MC simulation is not doing real world events and worst cases may pop up then could ever happen in the real economic world. Do they allow many overlapping series? I would imagine the design and the user choices is key to how good the simulator is.
It's true the FIRECalc has many sequences with common years like the 19291932 period. I find it best to just go for 100% success rate with a reasonable floor on the minimum portfolio value during the entire simulation. If FC fails, I want to know why and what years were responsible. That is where the spreadsheet output can be very handy.
I was plunking around and found this simple MC simulator from Vanguard. Notice there is no portfolio minimum value besides zero allowed: https://retirementplans.vanguard.com...estEggCalc.jsf
__________________



12072013, 09:12 AM

#7

Give me a museum and I'll fill it. (Picasso) Give me a forum ...
Join Date: Jun 2002
Location: Texas Hill Country
Posts: 39,201

Quote:
Originally Posted by photoguy
As others have mentioned FIRECALC reports on the exact number of runs in HISTORY that would have failed.
It doesn't produce what the probability of success is going forward (which is THE NUMBER that everyone wants). If you want to use the FIRECALC number as a forward estimate you should probably be concerned with two types of errors:
Bias which would be if the historical Firecalc success rate systematically under or overestimates the probability going forward. I think many people believe that Firecalc is biased to produce higher success rates (as a future estimate) because currently PE10 is high right now and bond yields are low. This is why so many people take a haircut on the allowable withdrawal rate (e.g., instead of 4% I'll use 3.5%).
Variance which would be due to fluctuations in the observed rate due to chance (but not systematic errors). For example, if you flipped a coin 10 times you might get 4 heads and estimate the probability of getting heads as 40%. In this case, due to variance the estimate is 10% too low.
Firecalc does not produce any sort of confidence bounds. To deal with this, you could mentally add +/ 10% to whatever number firecalc issues (I guess someone could calculate the impact of the overlapping periods by running simulations to figure out a better estimate than this).

You forgot the third type of "error":
Asteroid Strike which would likely result in portfolio failure regardless of whatever fudge factor you added to FIRECalc.
__________________
Numbers is hard.
Retired in 2005 at age 58, no pension



12072013, 09:28 AM

#8

Give me a museum and I'll fill it. (Picasso) Give me a forum ...
Join Date: Aug 2011
Location: West of the Mississippi
Posts: 5,174

Ah, yes. The exogenous events that throw the best plans under the bus.
The only way to deal with these events and their effect on FireCalc is to "Trust but Verify". And pivot if necessary.
__________________
The worst decisions are usually made in times of anger and impatience.



12072013, 09:39 AM

#9

Give me a museum and I'll fill it. (Picasso) Give me a forum ...
Join Date: Jan 2008
Location: Chicagoland
Posts: 10,663

Quote:
Originally Posted by REWahoo
You forgot the third type of "error":
Asteroid Strike which would likely result in portfolio failure regardless of whatever fudge factor you added to FIRECalc.

I always thought this (below) might be you...so you've moved from comet to asteroid?
Quote:
Originally Posted by FIRECALC
How can I be sure these results will work in the future?
FIRECalc's standard model uses the overall US stock market performance. Most 401k and similar retirement plans offer investment choices ("index funds") that are closely tied to the overall market performance, and the others generally tell you how they compare.
If the next few decades are even worse for the stock market than the worst that has ever been seen, including the Great Depression, then all bets are off.
But as one early retiree pointed out, there isn't much anyone can do to prepare for a comet hitting us!

__________________
No one agrees with other people's opinions; they merely agree with their own opinions  expressed by somebody else. Sydney Tremayne
Retired Jun 2011 at age 57
Target AA: 60% equity funds / 35% bond funds / 5% cash
Target WR: Approx 2.5% Approx 20% SI (secure income, SS only)



12072013, 09:47 AM

#10

Give me a museum and I'll fill it. (Picasso) Give me a forum ...
Join Date: Sep 2005
Location: Northern IL
Posts: 16,437

Quote:
Originally Posted by photoguy
...
Bias which would be if the historical Firecalc success rate systematically under or overestimates the probability going forward. I think many people believe that Firecalc is biased to produce higher success rates (as a future estimate) because currently PE10 is high right now and bond yields are low. This is why so many people take a haircut on the allowable withdrawal rate (e.g., instead of 4% I'll use 3.5%). ...

But...
That 4% succeeds in all but the worst 5% of the periods in the database. So I think the relevant question is whether the current market is more bloated then it was in those few periods.
We aren't really looking at averages, so I don't think it is relevant whether the market is more overheated than average, just if it is hotter than the 'worst' times of the past.
For reference, I use a 100% success rate and 45 years for my target. One of us may live that long, and I don't want to use a plan that has been known to fail in the past. Still, the future could be worse than the worst of the past. Hopefully, I recognize that, and adjust in time.
ERD50
__________________



12072013, 09:48 AM

#11

Moderator Emeritus
Join Date: Sep 2007
Posts: 15,369

You just have to keep your portfolio covered with a good grade of tin foil.
__________________
Go Cubs



12072013, 10:51 AM

#12

Thinks s/he gets paid by the post
Join Date: Jul 2011
Location: The Bay Area
Posts: 1,284

Quote:
Originally Posted by photoguy
As others have mentioned FIRECALC reports on the exact number of runs in HISTORY that would have failed.
It doesn't produce what the probability of success is going forward (which is THE NUMBER that everyone wants). If you want to use the FIRECALC number as a forward estimate you should probably be concerned with two types of errors:
Bias which would be if the historical Firecalc success rate systematically under or overestimates the probability going forward. I think many people believe that Firecalc is biased to produce higher success rates (as a future estimate) because currently PE10 is high right now and bond yields are low. This is why so many people take a haircut on the allowable withdrawal rate (e.g., instead of 4% I'll use 3.5%).
Variance which would be due to fluctuations in the observed rate due to chance (but not systematic errors). For example, if you flipped a coin 10 times you might get 4 heads and estimate the probability of getting heads as 40%. In this case, due to variance the estimate is 10% too low.
Firecalc does not produce any sort of confidence bounds. To deal with this, you could mentally add +/ 10% to whatever number firecalc issues (I guess someone could calculate the impact of the overlapping periods by running simulations to figure out a better estimate than this).

It's true that current PE10 is high historically, in the range if the historical highs. But, that's why we use >=95% success rates and analyze any failures. To call the current situation 'biased' seems to be counter to the methodology and overly pessimistic.
S&P 500 PE Ratio
Quote:
Originally Posted by REWahoo
You forgot the third type of "error":
Asteroid Strike which would likely result in portfolio failure regardless of whatever fudge factor you added to FIRECalc.

Revising The Math: Chances Of Asteroid Hitting Earth Greater Than Once Thought « CBS Baltimore
Evidently, not as remote a possibility as we previously thought. Damn it REW, now I have to change my spreadsheet.
" Figuring in those smaller asteroids, the airburst like the one in Russia has dropped from a possibility of once every 150 years to once in every 30 years."
However, if Congress was in session during such an event, this might cause a dramatic rise in portfolio values.
" It prompted NASA to do a quiet tabletop drill, where such a hit on D.C. was predicted to kill 78,000 people. An asteroid smaller than a football field might do it."
__________________
You may be whatever you resolve to be.



12072013, 11:08 AM

#13

Administrator
Join Date: Jan 2007
Location: New Orleans
Posts: 34,845

I don't think that any of the methods to help us determine withdrawal rate are reliable enough to bet my life and future on. What these methods (including FIRECalc) provide to me, is a starting point. Should I withdraw 20%, or 0.00001%? FIRECalc gives me a starting point in the decision making process and what I do with it depends on me.
FIRECalc is only telling us what happened in the past. It really says nothing about the future IMO. The assumption that the future would mirror the past in a probabalistic sense, is ours to make or to not make.
Perfect, no. Better than many other methods (including my crystal ball), yes.
__________________
"You can never cross the ocean until you have the courage to lose sight of the shore."    C. Columbus



12072013, 09:33 PM

#14

Give me a museum and I'll fill it. (Picasso) Give me a forum ...
Join Date: Dec 2004
Location: the City of Subdued Excitement
Posts: 5,247

Quote:
Originally Posted by Lsbcal
I don't know what is built into Monte Carlo simulators. Clearly MC simulation is not doing real world events and worst cases may pop up then could ever happen in the real economic world. Do they allow many overlapping series? I would imagine the design and the user choices is key to how good the simulator is.
It's true the FIRECalc has many sequences with common years like the 19291932 period. I find it best to just go for 100% success rate with a reasonable floor on the minimum portfolio value during the entire simulation. If FC fails, I want to know why and what years were responsible. That is where the spreadsheet output can be very handy.
I was plunking around and found this simple MC simulator from Vanguard. Notice there is no portfolio minimum value besides zero allowed: https://retirementplans.vanguard.com...estEggCalc.jsf

Cool! I had not seen this on Vanguard's web site. Thanks!
Visually, this shows how important the first years are.
Also, the importance of reevaluating after 4 or 5 years. Shortening the duration from 30 years to 25 years does not change the survivability very much. One 75/25/0 4.5% scenario showed 84% survivability in 30 years and 89% in 25 years.
Many nice tools these days.
__________________
my bumpersticker:
"I am not in a hurry.
I am retired.
And I don't care how big your truck is."



12082013, 04:29 AM

#15

Thinks s/he gets paid by the post
Join Date: Aug 2006
Posts: 2,241

Quote:
Originally Posted by photoguy
As others have mentioned FIRECALC reports on the exact number of runs in HISTORY that would have failed.
It doesn't produce what the probability of success is going forward (which is THE NUMBER that everyone wants). If you want to use the FIRECALC number as a forward estimate you should probably be concerned with two types of errors:
...
Variance which would be due to fluctuations in the observed rate due to chance (but not systematic errors). For example, if you flipped a coin 10 times you might get 4 heads and estimate the probability of getting heads as 40%. In this case, due to variance the estimate is 10% too low.
Firecalc does not produce any sort of confidence bounds. To deal with this, you could mentally add +/ 10% to whatever number firecalc issues (I guess someone could calculate the impact of the overlapping periods by running simulations to figure out a better estimate than this).

This is what I was getting at. Many financial planners (and others) use the 4% WR and 95% success probability as a predictor of a future probability even though they are based upon only about five independent 30year data sets, similar to your coinflip example. I haven't seen any rigorous research along the lines you suggest.
__________________
I'd rather be governed by the first one hundred names in the telephone book than the Harvard faculty  William F. Buckley



12082013, 07:33 AM

#16

Give me a museum and I'll fill it. (Picasso) Give me a forum ...
Join Date: Jan 2008
Location: Chicagoland
Posts: 10,663

Quote:
Originally Posted by FIRE'd@51
This is what I was getting at. Many financial planners (and others) use the 4% WR and 95% success probability as a predictor of a future probability even though they are based upon only about five independent 30year data sets, similar to your coinflip example. I haven't seen any rigorous research along the lines you suggest.

I think we agree with you. But there have been several "rigorous" studies about future withdrawal rates with the upshot typically "3% is the new 4%"  coincidentally there's another thread on the topic just started here. And I am sure there will be more research. Most factor in prolonged low bond yields and interest rates and the current above average equity market valuations to arrive at more conservative WR estimates. Even using 100% success rates in FIRECALC isn't an answer as your OP suggests.
But again FIRECALC does not predict the future, and that's very clearly stated. So FIRECALC doesn't "significantly overestimate" or give incorrect success probabilities. It simply plugnchugs history and spits out those results for consideration.
To fault an elegant "app" that the authors offer at no cost (few donate anything at all) for providing answers it doesn't pretend to offer may be undeserved. I tried to write my own FIRECALC, if you think it's easy, give it a whirl. But for anyone who still may erroneously think FIRECALC predicts anything, hopefully your thread will help them better understand.
Don't forget while a 95% success rate means you have a 1 in 20 chance of failing if you withdraw X%, it also means you have a 19 in 20 chance of withdrawing more than X%. And withdrawal rates are an axe, not a scalpel. You will undoubtedly have to make adjustments along the way...
__________________
No one agrees with other people's opinions; they merely agree with their own opinions  expressed by somebody else. Sydney Tremayne
Retired Jun 2011 at age 57
Target AA: 60% equity funds / 35% bond funds / 5% cash
Target WR: Approx 2.5% Approx 20% SI (secure income, SS only)



12082013, 08:15 AM

#17

Thinks s/he gets paid by the post
Join Date: Aug 2006
Posts: 2,241

Quote:
Originally Posted by Midpack
To fault an elegant "app" that the authors offer at no cost (few donate anything at all) for providing answers it doesn't pretend to offer may be undeserved

I am not faulting FireCalc. To the contrary, I think it's the best program around. I am only wondering if there is a way to manipulate the overlapping historical data to better simulate a statistical ensemble of "independent" data points. This has nothing to do with PE10 or other valuations, which will always be present. I believe these affect the mean of the exante distribution, but not the variance. I'm interested in focusing soley on the statistical analysis which, I believe, can give us a better handle on the variance, which ultimately determines the exante probabilities.
__________________
I'd rather be governed by the first one hundred names in the telephone book than the Harvard faculty  William F. Buckley



12082013, 08:21 AM

#18

Give me a museum and I'll fill it. (Picasso) Give me a forum ...
Join Date: Jan 2008
Location: Chicagoland
Posts: 10,663

Quote:
Originally Posted by FIRE'd@51
I am not faulting FireCalc. To the contrary, I think it's the best program around. I am only wondering if there is a way to manipulate the overlapping historical data to better simulate a statistical ensemble of "independent" data points. This has nothing to do with PE10 or other valuations, which will always be present. I believe these affect the mean of the exante distribution, but not the variance. I'm interested in focusing soley on the statistical analysis which, I believe, can give us a better handle on the variance, which ultimately determines the exante probabilities.

Fair enough, let us know if you find a more meaningful method for analyzing historical returns and withdrawals rates. I am sure many here would be interested...
__________________
No one agrees with other people's opinions; they merely agree with their own opinions  expressed by somebody else. Sydney Tremayne
Retired Jun 2011 at age 57
Target AA: 60% equity funds / 35% bond funds / 5% cash
Target WR: Approx 2.5% Approx 20% SI (secure income, SS only)



12082013, 08:29 AM

#19

Thinks s/he gets paid by the post
Join Date: Aug 2006
Posts: 2,241

Quote:
Originally Posted by Midpack
Fair enough, let us know if you find a more meaningful method for analyzing historical returns and withdrawals rates. I am sure many here would be interested...

Midpack
I find your tone a bit offensive. I started this thread with the hope that one (or more) of the thousands of readers of this forum better versed in statistics than myself would suggest a way to do this, or even if it can be done. If I knew how to do it myself, I would have done it already.
__________________
I'd rather be governed by the first one hundred names in the telephone book than the Harvard faculty  William F. Buckley



12082013, 08:37 AM

#20

Give me a museum and I'll fill it. (Picasso) Give me a forum ...
Join Date: Jan 2008
Location: Chicagoland
Posts: 10,663

Quote:
Originally Posted by FIRE'd@51
Midpack
I find your tone a bit offensive. I started this thread with the hope that one (or more) of the thousands of readers of this forum better versed in statistics than myself would suggest a way to do this, or even if it can be done. If I knew how to do it myself, I would have done it already.

Sorry about that, but you're reading in "tone" that's not intended. I tend to err on the side of being direct, sometimes that's helpful but I realize sometimes it's decidedly not helpful. We/I would be sincerely interested on better/other studies, but for whatever reason I felt your OP read as potentially undue criticism of FIRECALC as did other members. Evidently we both misread each other a bit...
__________________
__________________
No one agrees with other people's opinions; they merely agree with their own opinions  expressed by somebody else. Sydney Tremayne
Retired Jun 2011 at age 57
Target AA: 60% equity funds / 35% bond funds / 5% cash
Target WR: Approx 2.5% Approx 20% SI (secure income, SS only)





Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)


Thread Tools 
Search this Thread 


Display Modes 
Linear Mode

Posting Rules

You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
HTML code is Off




» Recent Threads













