Study Questions Retirement Calcuators

[Gone4Good]

This book made me look at all such new studies with a healthy dose of skepticism:

I think that's why you don't see anyone outside the media pay too much attention to any one study. Often times it takes multiple studies done over numerous years with a variety of different methodologies to arrive at a consensus conclusion among experts. And that conclusion may not even be reflected in a single study, but represent the cumulative findings of many.

Of course that is all too painstaking for the general public. As Ha says, "just gimme an answer" . . . especially one that confirms what I already believe.

 

[OrcasIslandBound]

There are some "good" calculators out there. I think FireCalc is one, Fidelity RIP tool is another. Consumer reports points out that the better calculators use historical data combined with Monte Carlo simulations to get the results of predicting success rates. However, the implied assumption is that history will repeat itself. Some history examples violate these assumptions however. Example, when a wheel barrow full of money wouldn't buy a loaf of bread in Germany during the 30's. If this happens again, we'll all be focused on what we happen to have in the cupboard and car.

Sent from my Nexus 4 using Early Retirement Forum mobile app

 

[rdy2go]

I had a friend in college that was finishing up his doctorate in statistics. he earned pocket money by doing the statistical work for other grad students for their thesis or dissertation work. He would ask them "what do you want the numbers to prove?" and would make the data fit whatever angle they were pursuing, knowing full well that no one would understand the statistical work presented. He could make the data say whatever he wanted it to say and no one was the wiser. He went on to work for NASA by the way.

 

[aggie76]

I think I found a simple solution to the issues presented in the study. Whenever I use the MANY MANY calculators I always plug in the absolute worse case ( within reason) inputs. I have inflation at 3%, I input portfolio return at 4%, life expect at age 95 for both of us ( not at ALL likely)....etc.... I figure that IF the various calcs give me a high probability under those conditions I should be relatively safe under historical norms.
Of course someone always brings up the Mad Max or the Weimer Rep scenarios....Simple...All that will count after the zombie or money wheelbarrows events will be my GUNS AND AMMO...I will save the last 2 rounds for my and wife and I as in..who the hell in their right mind wants to live in that world !!

 

[Midpack]

I think that's why you don't see anyone outside the media pay too much attention to any one study. Often times it takes multiple studies done over numerous years with a variety of different methodologies to arrive at a consensus conclusion among experts. And that conclusion may not even be reflected in a single study, but represent the cumulative findings of many.

Of course that is all too painstaking for the general public. As Ha says, "just gimme an answer" . . . especially one that confirms what I already believe.

+1.

That's why you have to take many media reports with a grain of salt, and why you see so many findings seemingly reversed in time. Most media outlets are more focused on getting your attention, than reporting news...

And some here seem to take that approach too, posting an article or study that provokes replies...but no substantial or new information.

 

[Andy2]

There is no perfect answer to this:
Most calculators provided by financial institutions are biased towards "encouraging" people to save more than they might need for longer as that suits their own financial interests.
Spreadsheets can be really good but are maintenance intensive and a bad formula can be invisible and skew the results
I personally do not like Monte Carlo simulations...

In my opinion you need to develop a plan that works for you based on a combination of the best tools you can find..

 

[Big_Hitter]

I personally do not like Monte Carlo simulations...

then how do you define "accuracy" with regard to retirement planning?

 

[papadad111]

I quickly scanned the research paper. My first question was "what's the gold standard?". Of course, it would be outcomes, i.e. portfolio survival of a large cohort of retirees over decades, compared with predicted results from the calculators prior to retirement. Obviously, that would be impossible to study at this time.

The methodology the authors used was to review the inclusion of predictive data inputs in the various calculators. They focus on retiree-specific variables (age, income, assets are categories). Inflation, stock and bond return rates are also considered. Many of the calculators tested did not include variables that have been predictive in previous research and that the authors and a group of FAs considered important.

They then designed a specific marginal scenario and ran it through all the calculators. The results were wildly inconsistent. It is not clear how much of the inconsistency was due to variation in input variables or variation in calculator algorithms.

FireCalc was among the calculators tested, but is not listed in the final table, so it is not possible to review its performance in the study. Omission of data from the results makes me concerned.

The conclusion is that the design of publicly available calculators needs to be improved to avoid leading people astray. This makes sense to me. Retirement calculators are relatively recent innovations and will continue to evolve. In particular, their predictive values need to be validated with post hoc data. While it is not perfect, I do think this was a worthwhile study, if only to remind us not to encourage someone to ER just because "FireCalc says you're good to go".

Sadly no calculator can predict the future. Not future returns. Not future life expectancy.

If one is conservative in their WR, stay in the low 3 percent range and holds an AA on order of 80/20, then they should be ok by all calculators.

If not, then the surprise would not have been predictive by any of the calculators anyway. (Enter asteroid...).

It's really the ones on the very margin that are at risk and then it really becomes a game of craps. Throwing the dice to see what happens. There will be some winners and some losers based on happenstance. That's life. That's Lady Luck.

The entire concept of retirement is but a generation...maybe two generations old. It's all new. Makes sense there are no well dialed calculators to predict the future.

Flux capacitor, anyone?

 

[MichaelB]

Sadly no calculator can predict the future. Not future returns. Not future life expectancy.

If one is conservative in their WR, stay in the low 3 percent range and holds an AA on order of 80/20, then they should be ok by all calculators.

The usefulness of a calculator is not because it predicts the future, it is in the way it frames and models the changes and flows of assets, income and expenses over time. The paper makes this point, but in a rather muddled way. Still, as Meadbh points out, their two recommendations are sensible. First, smoking as a proxy for overall health, and second, focusing on default parameters, both could make calculators more useful.

FIRECalc modeling shows that (when in the withdrawal phase) increasing the equity allocation above 40% - 50% does not significantly improve the probability of portfolio survival. Lowering the withdrawal rate from 4% to 3% does, but only slightly.

One characteristic of a useful calculator would be to measure and project the range of outcomes and impact on portfolio survival when changing these two parameters.

 

[Gone4Good]

The usefulness of a calculator is not because it predicts the future, it is in the way it frames and models the changes and flows of assets, income and expenses over time.

I think that is precisely right.

As a first pass you want a model to tell you whether you're plan is reasonable. But that is really only the first pass. A good calculator helps you think about and investigate what success actually means.

That is why FireCalc is so useful. It's not a black box. We know the assumptions. We can debate whether those assumptions are reasonable. And if we come to the conclusion that they're not, we can adjust our strategy to account for what we think of as the flaws in the model.

Another huge benefit of a calculator like FireCalc is that you can dig into the results. It's not enough for me to know that my plan was Successful when success is defined as having $1 left when I die.

FireCalc gave me the ability to follow one of the adverse market scenarios. And by doing that I got a good sense that what the model saw as a "successful" scenario (living on less than a quarter of your original assets for a decade or more, all the while bumping against financial ruin and hoping to die on schedule) wasn't even close to how I'd want to actually live.

Most calculators just give you a number. That's not nearly enough.

 

[Gone4Good]

FIRECalc modeling shows that (when in the withdrawal phase) increasing the equity allocation above 40% - 50% does not significantly improve the probability of portfolio survival. Lowering the withdrawal rate from 4% to 3% does, but only slightly.

If you only look at portfolio survival that definitely seems true. Moving from a 4% withdrawal over 30 years with a 60/40 AA to a 3% withdrawal increases portfolio survival from 95% to 100%.

That doesn't look like a big deal. But that's only because we're using a statistic that's capped out at 100% - any improvements beyond that are completely missed. If we look at the ending portfolio balances we get a much better sense of what that 100bp difference in withdrawal rate really means.

With a 4% WR on a $1MM portfolio the worst-case ending balance bottoms out at ($272K). At 3% WR my balance bottoms out at $561K. That's a huge difference. One strategy results in potential ruin while the other has a downside scenario where I still have more than half my original assets.

But that's not all. If we tell FireCalc that we want to maintain a $560K balance we learn that a 4% withdrawal fails to do so 45% of the time.

In other words, a 4% withdrawal strategy results in a portfolio that declines below the worst case scenario of a 3% withdrawal nearly half the time. That's an enormous difference and one that a simple "portfolio survival" metric completely misses.

To bring things back on-topic, the ability to do this kind of analysis is why FireCalc is such a great tool.

 

[samclem]

FireCalc gave me the ability to follow one of the adverse market scenarios. And by doing that I got a good sense that what the model saw as a "successful" scenario (living on less than a quarter of your original assets, bumping against financial ruin, hoping to die on schedule) wasn't even close to how I'd want to actually live.

+1. I try to imagine what it would be like, emotionally, to ride some of those "successful" lines in real life and with no ability to "peek ahead" and see that the line eventually takes an upward turn. It would be very uncomfortable.

 

[hnzw_rui]

The entire concept of retirement is but a generation...maybe two generations old. It's all new.

Hmm, my grandparents have been retired for as long as I can remember. I'm Asian so for me, multi-generational households are the norm. Of course in our case, the middle generation live in the family home and fund their parents retirement while the older generation are built-in babysitters so both husband and wife can go to work.

 

[photoguy]

Sadly no calculator can predict the future. Not future returns. Not future life expectancy.

Retirement calculators are all about predicting the future, either directly or indirectly. If they weren't relevant to our future, we wouldn't care about them. There are also tons of other statistical models that can predict future equity returns and life expectancy.

Now if by prediction, you mean providing an accurate point estimate like saying your $1M portfolio will be worth 1.56M in ten years. then yes retirement calculators have nowhere near this level of power and would be useless at this task.

However a calculator can do things like provide an expected outcome and the dispersion of results. E.g. on average the portfolio value is 1.5x but could vary as much as 0.5x to 2.5x. As others have noted, perhaps the most important thing about interpreting the outputs is understanding how much variance there can be in results.

 

[sengsational]

One characteristic of a useful calculator would be to measure and project the range of outcomes and impact on portfolio survival when changing these two parameters.

To address the inability of some calculators to perform a Monte-Carlo simulation, one can do 'sensitivity analysis' like this as an alternative. Monte Carlo tries a bunch of random things, and typically many of them don't make any sense (the market is not a uniform distribution, which is what is used much of the time). If you run many scenarios with various ranges of inputs, you can get a pretty good idea of what the range of possibilities would be.

 

[papadad111]

Hmm, my grandparents have been retired for as long as I can remember. I'm Asian so for me, multi-generational households are the norm. Of course in our case, the middle generation live in the family home and fund their parents retirement while the older generation are built-in babysitters so both husband and wife can go to work.

Yea. That's true. Asian culture, having lived in Asia for 25 years, is indeed different from western retirement culture. Multi family living in the USA is hardly the norm.

 

[Big_Hitter]

Hmm, my grandparents have been retired for as long as I can remember. I'm Asian so for me, multi-generational households are the norm. Of course in our case, the middle generation live in the family home and fund their parents retirement while the older generation are built-in babysitters so both husband and wife can go to work.

I'm partially funding my parents' retirement - it's called Social Security

I can't imagine living with my parents as an adult.

Well except for about 2 months when I was moving back to Houston and looking for a house. I actually lived with them in my 30s. When someone asked me where I lived and I said "with my parents" I instantly became the biggest loser in the room.

Back on topic - what happened to those threads where someone said they had an accurate calculator but couldn't comment on it? Holding out on us?

 

[Fedup]

I never fully trusted any retirement calculator. I ran it to get a rough idea. But also withholding some assets just in case I'm wrong. Who am I going to go after if my numbers are wrong when I'm in my 90s.


Sent from my iPad using Early Retirement Forum

 

[Fedup]

Yea. That's true. Asian culture, having lived in Asia for 25 years, is indeed different from western retirement culture. Multi family living in the USA is hardly the norm.

It's not universal, having lived in Asia for 10 years, I can tell you it varies from household to household. It's actually a very foreign concept for my family.


Sent from my iPad using Early Retirement Forum

 

[Bir48die]

Sure I did just about every free retirement calculator known to man. Sure they made me feel good. I also paid to have an independent study done. But, there are easy ways to set retirement scenarios up on Excel spreadsheets which I did for my own peace of mind. My breakdown was:

Retirement holdings at one point in time
One with early SS
Worst case scenario
Worst case scenario early SS

Overlayed that with my various budgets. Conservative returns of 4%. A bit obsessive but effective.

Now just waiting to exit.

 

[Markola]

Newsflash: Tenured faculty with lifetime appointments critique retirement calculators' failure to predict with 100% certainty the financial security of everyone else.