What Success % do you use?

[ERD50]

Only have a rough approximation from various longevity calculators, many of which conflict on various factors. Even FireCalc is based on shaky assumptions such as: the next 30 years will reflect the results from the 30 years before age 65. That is such a crazy assumption that the whole effort is a house of cards. ...

That is not what FIRECalc does! It reports every 30 year period in its history (age has nothing to do with it). That's way, way different, and far more useful. (ooops, cross posted with MB)

...
As investors have said elsewhere, we are in a time of unprecedented easy money thanks the Fed. That has never been the case before. The current generation cannot imagine what life was like in the 80s. (I held a bridge mortgage that peaked at 22%!)

Not sure what you mean by "the current generation". But many retirees and some considering retiring lived through the inflation of the 80's (I had a 17% mortgage, adjustable, so it ended well).

At any rate, anyone running FIRECalc will be tested against that 80's inflation. 1966 is one of the 'killer' starting years, largely due to the 80's inflation.


-ERD50

 

[Car-Guy]

When I was working I used every calculator I could find and still built my own spreadsheets. At that point in my life with a 30+ year time horizon, I always wanted to be at more than 100%. Now that I'm retired and getting older and my time horizon has become much shorter, I don't need any stink-en calculators to figure it out.

 

[OldShooter]

FIRECalc does not make any assumption regarding future returns. It just applies the portfolio and spending input to all the past historical market cycles and measures the survival rate.

True, but I think it's kind of nit picking. Certainly the vast majority of people using Firecalc are thinking of its "number" as being predictive. That, IMO, is the reason that the accurate-sounding number is seriously misleading. If the number was consistently interpreted by people as simply a backtesting result, then any number of digits is fine.

The text on the FireCalc page makes it clear that it is simply backtesting, even to the point of emphasizing it: " ... FIRECalc will tell you how often your strategy would have worked throughout history. ..." They can't make it much clearer than that. But reading the posts here it doesn't seem to me that people are paying attention to this. Maybe I'm wrong.

 

[NW-Bound]

I really like FIRECalc. It's a good start. It's like knowing the record heat and cold in your area, to allow for adequate air conditioning and heating for comfortable living.

But, but, but new record temperatures are still being set. Not every year, but once in a while. You can survive, but it will be uncomfortable. You can adapt. It does not have to be lethal.

 

[big-papa]

So let's say I'm programming a MonteCarlo app. Let's say it gives results that are more optimistic than history. Do I now 'tweak' the factors? How far do I go before someone says it is too negative? It all seems crazy to me.

Not everyone. That's why some choose a spending level below the 100% historical SWR. That is to allow for a future that is worse than the actual past. Plus, many have a 'Plan B'.

You are also conflating the concept of the tool predicting the future, and the concept of people applying the results to plan for the future.

Example:

Plan for the future: It gets cold here in winter, into the - 20 F range some years, and even down to -32 once. I better plan accordingly.

Predicting the future: I predict a low of -23F this winter.


-ERD50

There are many ways to write a MonteCarlo routine. Factor tweaking isn't required, but if you're going to use the statistics of past data to do this in the first place, then you should use as much as you can extract. If a Monte run gave y ou results that were more optimistic than what happened in the past, then I would be concerned that either an insufficient amount of input data was used (10 years of returns is statistically insignificant, for example), or that an insufficient number of Monte Carlo runs were made. I would expect a well written Monte Routine to have results that fully encompass past results, with some results more optimistic and some more pessimistic than the actual past. It would be the actual point of using it in the first place - to see what sort of results might happen beyond what actually happened in the past.

Point taken that all people aren't taking the results from tools such as Firecalc that use actual sequences of returns verbatim as their plan, but they are using those results, even if they're discounting the results to provide a certain amount of additional safety. Is it enough? Nobody knows. A well written Monte will go beyond what the past did, but is what it shows enough? Again, nobody knows.

I'm actually not conflating anything - I'm only arguing about the validity of MonteCarlo as an additional tool. What one does with the results of it or of a backtest that uses sequences of returns as they actually happened is totally up to the individual. The only thing I can be reasonably certain about is that the future will not look like the past. I can deal with that by being conservative, not knowing whether it's conservative enough, and/or I can deal with it by being flexible as my retirement progresses. But I'm always to at least start things off with data that is based on the past in some way.

 

[Midpack]

True, but I think it's kind of nit picking. Certainly the vast majority of people using Firecalc are thinking of its "number" as being predictive. That, IMO, is the reason that the accurate-sounding number is seriously misleading. If the number was consistently interpreted by people as simply a backtesting result, then any number of digits is fine.

The text on the FireCalc page makes it clear that it is simply backtesting, even to the point of emphasizing it: " ... FIRECalc will tell you how often your strategy would have worked throughout history. ..." They can't make it much clearer than that. But reading the posts here it doesn't seem to me that people are paying attention to this. Maybe I'm wrong.

I hope it’s not “the vast majority” but I’m sure some people think FIRECALC is predictive - that’s simply willful ignorance. As you note, the text makes it clear it’s just backtesting - the user has to project how to use history for their future.

I’m amazed that anyone could be so cavalier about retirement planning as to use FIRECALC (or any tool) without reading the text that accompanies it. Again, that’s willful ignorance.

And after about 10 years on this site, I’m still amazed at how misunderstood SWR is even here. The term and concepts are misused by some every day even here. It’s no wonder readers are still confused anew, but this forum is better informed than most.

 

[MichaelB]

True, but I think it's kind of nit picking. Certainly the vast majority of people using Firecalc are thinking of its "number" as being predictive. That, IMO, is the reason that the accurate-sounding number is seriously misleading. If the number was consistently interpreted by people as simply a backtesting result, then any number of digits is fine.

The text on the FireCalc page makes it clear that it is simply backtesting, even to the point of emphasizing it: " ... FIRECalc will tell you how often your strategy would have worked throughout history. ..." They can't make it much clearer than that. But reading the posts here it doesn't seem to me that people are paying attention to this. Maybe I'm wrong.

Not a nit to me, and there's no evidence to assume that most people believe FIRECalc to be a predictive model. If anything, just the opposite. Most discussions here that involve FIRECalc show most members understand it's purpose, design, and limitations.

Backtesting is testing a predictive model based on historical data. FIRECalc does not do this - it does not predict future returns. It takes user supplied personal financial data and quantifies portfolio survival rates across all historical time periods.

It does allow users to model and test different future return assumptions, but this is not part of it's core functionality.

 

[OldShooter]

I hope it’s not “the vast majority” but I’m sure some people think FIRECALC is predictive ...

Well, I dunno. Reading the posts here it certainly seems to me that people are looking for predictions. Otherwise, why would they be obsessing about the difference between 95 and 100 or even 80 and 100?

 

[bada bing]

I'm not retired yet, about 2 years left. I've been playing around with all the
online calculators. I like Firecalc and Fidelity RIP, although the percentage
score they output means very different things.

I think I will be comfortable going into retirement with a higher Firecalc failure rate
projection than most here seem to target. Somewhere around 85%-90% success
lets me start spending enough for some bucket list items. I could pare my
spending down where it would reach 100% on Firecalc, but it would be a lot
less fun.

A couple reasons I think I'll be comfortable with a lower success% than most
here target:
1. I have no requirement for a legacy. I expect there will probably be some
money left when I go, but it has no real value to me. I am happily single
and have no kids. Whatever legacy I leave will be for charity.

2. I will manage the potential failure case differently than others here have
mentioned. I'll be monitoring my portfolio balance and also the market cost
of a private annuity that will fill in my required spending gaps. If my
portfolio ever starts a slide to failure, I will seriously consider bailing out and
buying a SPIA before the balance slides too low. The concept is fleshed out
in Die Broke Basically it is a way to prioritize spending rather than portfolio balance.

 

[MarieIG]

100%

 

[big-papa]

For Firecalc users that like an SWR type of method, there's another tool out there that you might want to try - It's the Simba spreadsheet located over on Bogleheads. It was originally written just for the accumulation phase, but the current owner, siamond, has added withdrawal rate calculations in there as well.

Reasons I like it:
It's completely open, you can check every single calculation on the spreadsheet if you're so inclined.

All sources of data are shown

The current owner is more than willing to add new features (within reason).

It runs in Excel as well as in Google Sheets.

It's free.

It has many more asset classes available than Firecalc does. Note carefully though that not all asset have existed for the same period of time. There is some error detection if you try to backtest back to, say, 1927 with an asset that has only existed since 1970, for example.

It will give you the exact WR that was 100% safe for each starting year and it will plot it.

The author updates it at least every January with the previous year's returns results for each asset type.

The link to the link where you can download the latest version.
https://www.bogleheads.org/forum/viewtopic.php?f=10&t=2520&start=700#p2753812

Big-Papa

 

[TwoByFour]

I've never seen the value of MonteCarlo for this (it has its place in other areas).

How much variance to use for each component? Correlation between components? Seems like the people who promote them will say, well we adjust until it matches to the historical reports! What's the point? Seems circular to me.

-ERD50

That is exactly my concern with Monte Carlo - there is definitely an autocorrelation of equity returns ("momentum") so that one year is not independent from the previous year's returns, and there is a cross-correlation between inflation and equity returns (equities tend to be a good inflation hedge) and also between bonds and inflation, although with a lag.

So, to properly account for all of these you need not only a proper model for the statistical distributions of equity and bond returns, but also for all the cross- and auto-correlations. At that point you may as well just use the historical data!

It is not surprising that MC gives a different result than a historical RIP tool, but it is not possible to say that it is a more accurate model for the future. It is still just a guess.

My feeling is that too much credence is given to any RIP tool output, especially the difference between say, 90% and 100% success probability.

 

[NW-Bound]

Yes. More than one poster have brought up the fact that MC models have to be "calibrated" against something, and that something is historical data.

Anybody can create a Monte Carlo simulation with horrific models; worst cases with stocks crashing to 1% of peak values, hyperinflation like that of Zimbabwe that required circulation of 1-trillion-dollar bills, etc... Well, why not throw in an asteroid strike too? Everything is possible, right?

So, I'd rather take FIRECalc which has the actual historical data, then adjust the number myself. This way, I know that I have 25% or 50% extra margin, or whatever I want.

I have not used any Monte Carlo simulation. Do they tell you that "this simulation has 25% or 50% extra margin built-in" or something like that? And if they do, what is the basis for that claim? Past historical data?

See what I mean?

 

[big-papa]

That is exactly my concern with Monte Carlo - there is definitely an autocorrelation of equity returns ("momentum") so that one year is not independent from the previous year's returns, and there is a cross-correlation between inflation and equity returns (equities tend to be a good inflation hedge) and also between bonds and inflation, although with a lag.

So, to properly account for all of these you need not only a proper model for the statistical distributions of equity and bond returns, but also for all the cross- and auto-correlations. At that point you may as well just use the historical data!

It is not surprising that MC gives a different result than a historical RIP tool, but it is not possible to say that it is a more accurate model for the future. It is still just a guess.

My feeling is that too much credence is given to the RIP tool output, especially the difference between say, 90% and 100% success probability.

Yep, Monte or backtesting using historical return sequences - only allow us to see the future, "through a glass, darkly" . See posting I made above: Jim Otar attempted to put all of the items you mention above into his Monte spreadsheet, but cheap old me doesn't want to spend money on it. And the engineer in me wants to know to the last detail exactly HOW he accounted for each of those things. Otherwise, it's on my future "to do" list to write one for myself. I do believe that Monte provides yet another perspective of what sort of possibilities MIGHT happen beyond what actually happened in the past and is a good addition to other methods.

Not sure this has been brought up recently, but years ago when SWR methods were first being bashed, people were pointing out that despite the historical record going back to the late 1800's, there are only a finite number of 30 (or 40 or whatever) unique return sequences that don't overlap with any other. A similar concern to me is that some starting years participate less in the calculation of the % of successful outcomes than others. For example, suppose we have data that only goes back to 1930. We might look at a 30 year retirement that starts in 1930. Then a 30 year retirement that starts in 1931, etc. Note that the starting year 1930 only participates in the calculation one time. 1931 participates twice: a retirement starting in 1930 and again for a retirement starting in 1931. Etc. And then when you get to the end of the historical data, you have the same issue with the last year in the historical record only showing up once in the calculation, the second to last year in the historical record showing up twice, etc....

Some attempts were made by some individuals to "wrap" the data around from the last year of the historical record back to the first year such that every starting year contributed equally. But this sequence of returns never actually happened. A properly written MonteCarlo program can address this, but again, we will never have enough data to make it foolproof either.

Bottom line: there will never be enough data until t->infinity for either MonteCarlo or a pure in-sequence backtested SWR method to be foolproof as a view into the future. I don't plan on living that long.

 

[big-papa]

Yes. More than one poster have brought up the fact that MC models have to be "calibrated" against something, and that something is historical data.

Anybody can create a Monte Carlo simulation with horrific models; worst cases with stocks crashing to 1% of peak values, hyperinflation like that of Zimbabwe that required circulation of 1-trillion-dollar bills, etc... Well, why not throw in an asteroid strike too? Everything is possible, right?

So, I'd rather take FIRECalc which has the actual historical data, then adjust the number myself. This way, I know that I have 25% or 50% extra margin, or whatever I want.

I have not used any Monte Carlo simulation. Do they tell you that "this simulation has 25% or 50% extra margin built-in" or something like that? And if they do, what is the basis for that claim? Past historical data?

See what I mean?

The simplest monte carlo routines will look at the historical data derive the mean and standard deviations. Extra points if they recognize that stock returns don't look like a normal distribution, but something more like lognormal. More points can be added to add correlations between asset classes and auto-correlations within an asset class. None of this is really a calibration, it's more of an extraction. When you get into something like calibration is when you start to look at some of the things that Otar added which is different auto-correlations that lasted N years. He also added a Japan-like scenario if memory serves me.

Another method I've seen is to use bootstrapping. These types of simulators use actual annual returns not derived from the statistics of the entire return data set, at least partially. It's sort of assuming that the only returns for a given asset that could happen are the ones that have already happened.

Anyway, to my way of thinking, the purpose of a Monte Carlo simulation is to see what might happen beyond what actually happened in the past. You could just restrict yourself to the data or statistics of only US assets and you should be able to get some reasonable data beyond what might have actually happened in the past. But I would expect the results to fully encompass past results. I think that's a good way to augment data that comes from something like Firecalc or Simba. Or you could go further and add Japan-like (or Zimbabwe) or whatever type of scenarios as well. But if you can find the data, you can also do likewise for an SWR type of simulation - I've come across some articles where folks have done this.

 

[Dtail]

That is exactly my concern with Monte Carlo - there is definitely an autocorrelation of equity returns ("momentum") so that one year is not independent from the previous year's returns, and there is a cross-correlation between inflation and equity returns (equities tend to be a good inflation hedge) and also between bonds and inflation, although with a lag.

So, to properly account for all of these you need not only a proper model for the statistical distributions of equity and bond returns, but also for all the cross- and auto-correlations. At that point you may as well just use the historical data!

It is not surprising that MC gives a different result than a historical RIP tool, but it is not possible to say that it is a more accurate model for the future. It is still just a guess.

My feeling is that too much credence is given to any RIP tool output, especially the difference between say, 90% and 100% success probability.

Not sure you are referring to the Fidelity RIP calculator. Even if the score is at 100% of expenses covered(100 score), it reflects the expenses are covered in 90% of the scenarios. i.e. there isn't a true 100% success rate like Firecalc FWIW.

 

[Fred123]

That is exactly my concern with Monte Carlo - there is definitely an autocorrelation of equity returns ("momentum") so that one year is not independent from the previous year's returns, and there is a cross-correlation between inflation and equity returns (equities tend to be a good inflation hedge) and also between bonds and inflation, although with a lag.

So, to properly account for all of these you need not only a proper model for the statistical distributions of equity and bond returns, but also for all the cross- and auto-correlations. At that point you may as well just use the historical data!

It is not surprising that MC gives a different result than a historical RIP tool, but it is not possible to say that it is a more accurate model for the future. It is still just a guess.

My feeling is that too much credence is given to any RIP tool output, especially the difference between say, 90% and 100% success probability.

Exactly! Was just going to post same, but you beat me to it.


Some time ago I wrote myself a program that bootstraps from past equity/bond returns. Results are definitely more pessimistic than FireCALC, but not dramatically so.

 

[big-papa]

Exactly! Was just going to post same, but you beat me to it.


Some time ago I wrote myself a program that bootstraps from past equity/bond returns. Results are definitely more pessimistic than FireCALC, but not dramatically so.

And that's exactly what I would expect! The past isn't the only scenario that could ever happen, but there is a good amount of past data available. One of the other things about MonteCarlo is that each time you run it, you should get slightly different answers - based on that you can get a "confidence" number. One can then derive maybe a 98% SWR success rate with 95% confidence or some such.... Though the larger a Monte Run (more samples), the less the final results are going to vary...

 

[Dtail]

And that's exactly what I would expect! The past isn't the only scenario that could ever happen, but there is a good amount of past data available. One of the other things about MonteCarlo is that each time you run it, you should get slightly different answers - based on that you can get a "confidence" number. One can then derive maybe a 98% SWR success rate with 95% confidence or some such....

Bolded - that's another thing with Fidelity's calculator. Even though it is stated that is run off 250 random calculations, the results are the same when run twice in a row with no changes in input.

 

[big-papa]

Bolded - that's another thing with Fidelity's calculator. Even though it is stated that is run off 250 random calculations, the results are the same when run twice in a row with no changes in input.

Same here: Yet in their disclaimer they say: "Your results may vary with each use and over time."

When I run it, it says that it's doing 250 calculations. Again, I'd have to write my own and see if 250 runs is enough to converge to less than, say 1% in their final number. Or it could be that there is more uncertainty if your score is down around 50 or than it is when you're up in the 90's...

 

[USGrant1962]

I've always found the concept of success percentage in FIRECalc a little wonky. As discussed above, your percentage changes with the number of years tested simply because the divisor is changing - really nothing to do with your "real" success rate.

A percentage success rate makes a little more sense in Monte Carlo if you use hundreds or thousands of runs.

I tend to look at 100% in FIRECalc as "yes, I would have survived the Great Depression and the Great Inflation - for me that's good to go!"

 

[NW-Bound]

Monte Carlo simulation method is highly popular in engineering, and used with reasonable success. It is used to look at the cumulative effect of many random variables (in the 100s or 1000s), which are independent yet all affect the system in a complex manner.

Suppose you want to study the performance of an automatic landing system of an aircraft, and know the risk of a hard landing of a certain magnitude. That depends on the aircraft landing weight (what it carries), its center of gravity, the air density, the sudden wind gust that might happen, the glide slope angle, the landing speed, wind speed, small errors of the hundreds of onboard aircraft sensors (which are still within specs), etc... All these variables are independent of one another and they all affect the result, which you try to determine. Each of these variables can be isolated and studied, and possibly individually controlled. Even with environmental factors that cannot be controlled such as air temperature and wind speed, you can measure it and decide not to land and to divert to elsewhere.

The financial world is much more complex and all its variables are intertwined. It is driven by discrete random events that are not amenable to isolation, calibration, and control like an aircraft sensor accuracy such as a radar altimeter that is individually tested to a certain precision requirement.

 

[Fred123]

Same here: Yet in their disclaimer they say: "Your results may vary with each use and over time."

When I run it, it says that it's doing 250 calculations. Again, I'd have to write my own and see if 250 runs is enough to converge to less than, say 1% in their final number. Or it could be that there is more uncertainty if your score is down around 50 or than it is when you're up in the 90's...

I get a lot more variance with only 250 runs in my program. I find 2000 to be a good number for reasonable consistency.

 

[big-papa]

Monte Carlo simulation method is highly popular in engineering, and used with reasonable success. It is used to look at the cumulative effect of many random variables (in the 100s or 1000s), which are independent yet all affect the system in a complex manner.

Suppose you want to study the performance of an automatic landing system of an aircraft, and know the risk of a hard landing of a certain magnitude. That depends on the aircraft landing weight (what it carries), its center of gravity, the air density, the sudden wind gust that might happen, the glide slope angle, the landing speed, wind speed, small errors of the hundreds of onboard aircraft sensors (which are still within specs), etc... All these variables are independent of one another and they all affect the result, which you try to determine. Each of these variables can be isolated and studied, and possibly individually controlled. Even with environmental factors that cannot be controlled such as air temperature and wind speed, you can measure it and decide not to land and to divert to elsewhere.

The financial world is much more complex and all its variables are intertwined. It is driven by discrete random events that are not amenable to isolation, calibration, and control like an aircraft sensor accuracy such as a radar altimeter that is individually tested to a certain precision requirement.

Are you an Engineer? There are several of us on this forum. For me, it's Electrical Engineering - Semiconductor. And, yep, we use MonteCarlo all the time in our analysis....

 

[cyber888]

I'm Ok with 99% success at age 95, because there's a 95% probability of death at that age.

And if I'm still alive at at 95, I'll just sell my house for $700,000-$800,000 which will hold me up to 105 years old, where probability of death is at 99.9 %