Join Early Retirement Today
Reply
 
Thread Tools Search this Thread Display Modes
Study Questions Retirement Calcuators
Old 02-20-2016, 03:50 PM   #1
Recycles dryer sheets
 
Join Date: May 2015
Location: NorCal
Posts: 74
Study Questions Retirement Calcuators

In today's (2/20/16) print version of the WSJ - Study Questions Retirement Calculators

"...A new academic study of 36 such tools - many of them widely used - concludes that "in most cases, the available offerings are extremely misleading" and generally not helpful to consumers trying to figure out if they will have enough money to cover their expenses for the rest of their lives...The study was conducted by three researchers at Texas Tech University, including well-know financial adviser Harold Evensky...The researchers didn't identify the 11 tools that passed their test..."

Too late. I'm already out of the rat race...

New Study Questions Retirement Planning Calculators’ Accuracy - WSJ
__________________

__________________
FIREd_2015 is offline   Reply With Quote
Join the #1 Early Retirement and Financial Independence Forum Today - It's Totally Free!

Are you planning to be financially independent as early as possible so you can live life on your own terms? Discuss successful investing strategies, asset allocation models, tax strategies and other related topics in our online forum community. Our members range from young folks just starting their journey to financial independence, military retirees and even multimillionaires. No matter where you fit in you'll find that Early-Retirement.org is a great community to join. Best of all it's totally FREE!

You are currently viewing our boards as a guest so you have limited access to our community. Please take the time to register and you will gain a lot of great new features including; the ability to participate in discussions, network with our members, see fewer ads, upload photographs, create a retirement blog, send private messages and so much, much more!

Old 02-20-2016, 04:06 PM   #2
Thinks s/he gets paid by the post
 
Join Date: Apr 2005
Location: Midwest
Posts: 1,572
Quote:
Originally Posted by FIREd_2015 View Post
In today's (2/20/16) print version of the WSJ - Study Questions Retirement Calculators

"...A new academic study of 36 such tools - many of them widely used - concludes that "in most cases, the available offerings are extremely misleading" and generally not helpful to consumers trying to figure out if they will have enough money to cover their expenses for the rest of their lives...The study was conducted by three researchers at Texas Tech University, including well-know financial adviser Harold Evensky...The researchers didn't identify the 11 tools that passed their test..."

Too late. I'm already out of the rat race...

New Study Questions Retirement Planning Calculators’ Accuracy - WSJ
Wouldn't this undermine the entire concept of 401k's and related retirement veehickles?
__________________

__________________
razztazz is online now   Reply With Quote
Old 02-20-2016, 04:08 PM   #3
Thinks s/he gets paid by the post
 
Join Date: Mar 2009
Posts: 1,437
Too late here also. Now that I've given notice the last thing I need is to find out the calculators I've used are suspect. I figured it out for myself for the last 38 years, hopefully I'll still be able to do the math myself in the future.
__________________
Retired in 2016. Living off dividends / interest and a mini pension. Freedom.
foxfirev5 is offline   Reply With Quote
Old 02-20-2016, 04:10 PM   #4
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
pb4uski's Avatar
 
Join Date: Nov 2010
Location: Vermont & Sarasota, FL
Posts: 16,458
That's interesting in that when I was deciding to retire I ran just about every calculator that I could find with as close to the same assumptions that I could since they each ask for different inputs and the results didn't seem that different to me.... all gave me different versions of a green light.

IIRC others have had similar experience. If that is so, how could this study conclude they are generally misleading and unuseful?

Edited to add: just found the study... need to read http://papers.ssrn.com/sol3/papers.c...act_id=2732927
__________________
If something cannot endure laughter.... it cannot endure.
Patience is the art of concealing your impatience.
Slow and steady wins the race.
pb4uski is online now   Reply With Quote
Old 02-20-2016, 04:36 PM   #5
Recycles dryer sheets
 
Join Date: May 2015
Location: NorCal
Posts: 74
Quote:
Originally Posted by pb4uski View Post
Thanks for finding the study!
__________________
FIREd_2015 is offline   Reply With Quote
Old 02-20-2016, 04:37 PM   #6
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
pb4uski's Avatar
 
Join Date: Nov 2010
Location: Vermont & Sarasota, FL
Posts: 16,458
Don't bother. This is an idiotic study. The authors seem to think that whether one is a smoker or non-smoker is important variable. Reading between the lines it seem they think the planning time horizon should be commensurate with health and since smoking has an impact on health then it is important to the retirement planning time horizon. However, there are many very old smokers out there who defy the odds.... including my 85 yo DM.

Am I misreading it? I concede that I just breezed through it but I'm not about to waste time on something so preposterous.

Quote:
We find it interesting that “smoker vs. non-smoker” fell outside the “important” or higher importance classifications given the overwhelming evidence in empirical research of the strong negative impact smoking has on an individual’s longevity. In addition, Figure three shows that there is a noticeable gap in responses for the “smoker vs. non-smoker” and “personal health” variables among financial planning professionals. In aggregate, 43% of the respondents believe the variable “personal health” is “extremely important,” compared to 28% of the respondents believe it is “important,” one level from being classified as “not applicable.” In contrast, 45% of the respondents believe a tobacco use variable, such as smoker vs non-smoker, is “important” with only 29% of the respondents believing it is “extremely important.” Given the significant potential impact on mortality estimates, we are surprised that three times as many respondents believe a tobacco use variable is “not applicable” to building a retirement plan, as compared to respondents indicating “personal health” is “not applicable.” We wonder if the respondents considered the “personal health” variable a more inclusive variable.
__________________
If something cannot endure laughter.... it cannot endure.
Patience is the art of concealing your impatience.
Slow and steady wins the race.
pb4uski is online now   Reply With Quote
Old 02-20-2016, 05:01 PM   #7
Thinks s/he gets paid by the post
Tadpole's Avatar
 
Join Date: Jul 2004
Posts: 1,174
Yea, you can look at all these calculators alone. I choose to believe the sigma/sqrt(N) sort of stuff. So if they all give me a good retirement prospectus, the final distribution of my retirement prospect is quite narrow (or so I choose to believe until any opposing real data comes along and then it will be a bit too late).
__________________
Tadpole is offline   Reply With Quote
Old 02-20-2016, 05:49 PM   #8
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
 
Join Date: Jul 2006
Posts: 11,019
I quickly scanned the research paper. My first question was "what's the gold standard?". Of course, it would be outcomes, i.e. portfolio survival of a large cohort of retirees over decades, compared with predicted results from the calculators prior to retirement. Obviously, that would be impossible to study at this time.

The methodology the authors used was to review the inclusion of predictive data inputs in the various calculators. They focus on retiree-specific variables (age, income, assets are categories). Inflation, stock and bond return rates are also considered. Many of the calculators tested did not include variables that have been predictive in previous research and that the authors and a group of FAs considered important.

They then designed a specific marginal scenario and ran it through all the calculators. The results were wildly inconsistent. It is not clear how much of the inconsistency was due to variation in input variables or variation in calculator algorithms.

FireCalc was among the calculators tested, but is not listed in the final table, so it is not possible to review its performance in the study. Omission of data from the results makes me concerned.

The conclusion is that the design of publicly available calculators needs to be improved to avoid leading people astray. This makes sense to me. Retirement calculators are relatively recent innovations and will continue to evolve. In particular, their predictive values need to be validated with post hoc data. While it is not perfect, I do think this was a worthwhile study, if only to remind us not to encourage someone to ER just because "FireCalc says you're good to go".
__________________
Meadbh is offline   Reply With Quote
Old 02-20-2016, 06:06 PM   #9
Thinks s/he gets paid by the post
 
Join Date: Feb 2007
Posts: 1,905
During the mid to late 1990's when I first started thinking about ER I ran the calculators available at that time and I also got financial plans from a couple of independent Financial advisor’s at Ernst & Young and such, At the time they all said I was good to go and I ER'd end of 2002 at 52. Surprisingly, the most accurate forecast so far is one I ran in 1996 using QFP which back then had my liquid NW 20 years latter (in 2016) within 10% of what it actually is. I hope it continues to be accurate because it shows some very nice numbers for the 30th and 40th anniversaries...
__________________
ejman is offline   Reply With Quote
Old 02-20-2016, 06:10 PM   #10
Full time employment: Posting here.
 
Join Date: Mar 2015
Location: philly
Posts: 876
So let's see....
1) financial advisers lead people astray
2) calculating tools lead people astray
3) most Americans are financially illiterate and don't know enough on their own.

Let's see if I save money, stress from above, If I don't save any money stress that I can never retire.


I'm a newbie at this and in the 16 months I've been preparing myself (or at least attempting to) I think I've figured it out. Just like the latest food warnings, no matter what I do, if I wait long enough some one will come along and tell me to start worrying because it's wrong.

Glad I'm going to Vegas next month, think I'll upgrade my hotel to a suite and start enjoying spending. lol, at least when someone says it's wrong I can say I had a good time.

Jeez
__________________
bclover is offline   Reply With Quote
Old 02-20-2016, 06:51 PM   #11
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
pb4uski's Avatar
 
Join Date: Nov 2010
Location: Vermont & Sarasota, FL
Posts: 16,458
Quote:
Originally Posted by ejman View Post
During the mid to late 1990's when I first started thinking about ER I ran the calculators available at that time and I also got financial plans from a couple of independent Financial advisor’s at Ernst & Young and such, At the time they all said I was good to go and I ER'd end of 2002 at 52. Surprisingly, the most accurate forecast so far is one I ran in 1996 using QFP which back then had my liquid NW 20 years latter (in 2016) within 10% of what it actually is. I hope it continues to be accurate because it shows some very nice numbers for the 30th and 40th anniversaries...
Your post caused me to go back and look at my actual to expected. My end of 2015 portfolio value is 17% higher than what I expected when I retired at the end of 2011, totally due to better than expected investment returns (9% vs 5%).
__________________
If something cannot endure laughter.... it cannot endure.
Patience is the art of concealing your impatience.
Slow and steady wins the race.
pb4uski is online now   Reply With Quote
Old 02-20-2016, 06:55 PM   #12
Moderator
rodi's Avatar
 
Join Date: Apr 2012
Location: San Diego
Posts: 8,817
This study definitely seemed biased or incomplete. As pb4uski pointed out - they seemed obsessed with tobacco use and it's impact on longevity... But most calculators don't predict how long you'll live. For example Firecalc asks you to input the length of retirement (putting the burden on the user... )

I would have liked to see the scenario data and results from the 36 calculators. It wasn't there.

Folks on this board tend to be fairly well educated about the possible pitfalls of most retirement plans and take steps to mitigate - or at least consider the steps they might take if needed. I suspect that the vast majority of folks not on this board do not have the knowledge, understanding, or really care that much about these nuances... and if their simple calculator or 4% rule of thumb says they are good to go, they figure they are good to go.
__________________
Retired June 2014. No longer an enginerd - now I'm just a nerd.
micro pensions 7%, rental income 18%
rodi is offline   Reply With Quote
Old 02-20-2016, 06:56 PM   #13
Thinks s/he gets paid by the post
Car-Guy's Avatar
 
Join Date: Aug 2013
Location: Citizen of Texas
Posts: 2,483
I "pity the fool" who would retire solely based on one or more of these on-line calculators (including FIREcal). I think many are interesting and fun to use, however, I actually trusted my custom spreadsheets and common sense a lot more to help me make my decision to retire.
__________________
Car-Guy is offline   Reply With Quote
Old 02-20-2016, 10:45 PM   #14
Recycles dryer sheets
 
Join Date: Jul 2013
Posts: 189
I hope you are being sarcastic and not calling a lot of people fools.


I "pity the fool" who would retire solely based on one or more of these on-line calculators (including FIREcal).
__________________
alaska55 is offline   Reply With Quote
Old 02-21-2016, 06:37 AM   #15
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
Midpack's Avatar
 
Join Date: Jan 2008
Location: Chicagoland
Posts: 11,978
That was pretty vague and therefore useless. Some retirement calculators are misleading (definitely true, they're pretty easy to recognize), so you can't trust any of them. Thanks for the thoughtful insight.

Some Company ABC retirement calculators are black boxes that don't reveal underlying assumptions and their purpose is to sell you Company ABC investments and/or financial advice - really, isn't that shocking! Anyone who plugs in a few numbers without any idea what the underlying assumptions and calculations are, and believes the output is an idiot. Caveat emptor.

They list a bunch of calculators in Appendix A, but never provide any indication re: how useful any of them are/aren't. The results are highly variable, so you can't trust any of them. That would be like Consumer Reports testing all automobiles, finding some more reliable than others, and concluding you can't rely on cars. Huh?

And they say you can't rely on financial advisors either.

So work until you drop? Or take up smoking?

I guess if your a smoker you probably have enough, and if you're not you'll probably outlive your nest egg.

The paper wasn't at all value added for me. YMMV
__________________
No one agrees with other people's opinions; they merely agree with their own opinions -- expressed by somebody else. Sydney Tremayne
Retired Jun 2011 at age 57

Target AA: 60% equity funds / 35% bond funds / 5% cash
Target WR: Approx 2.5% Approx 20% SI (secure income, SS only)
Midpack is online now   Reply With Quote
Old 02-21-2016, 08:15 AM   #16
Thinks s/he gets paid by the post
Senator's Avatar
 
Join Date: Feb 2014
Location: Eagan, MN
Posts: 3,061
I had to re-read the study. A lot what was said had to do with user input. Some people did not know how to input the data, some calculators did not allow enough data, and smokers die faster. I also got the impression they were advocating for pensions.

Conclusion, There is no 100% way to predict what will actually happen.

At the end is a great list of calculators all with hyperlinks. I tried out a few, and they are basically the same. If that is all you get out of this study, it is worth it.

You can use historical results (FireCalc), you can use Monte Carlo, straight-line, and some sort of blend. There really is not much else.

I tried numerous calculators over the past few years, and none can tell me with 100% certainly I will be OK. They might give a statistical probability of 100%, but no one can ever say until the absolute end, "It worked". It is a leap of faith, similar to getting in your car and expecting to arrive back at home in one piece.
__________________
FIRE no later than 7/5/2016 at 56 (done), securing '16 401K match (done), getting '15 401K match (done), LTI Bonus (done), Perf bonus (done), maxing out 401K (done), picking up 1,000 hours to get another year of pension (done), July 1st benefits (vacation day, healthcare) (done), July 4th holiday. 0 days left. (done) OFFICIALLY RETIRED 7/5/2016!!
Senator is online now   Reply With Quote
Old 02-21-2016, 08:23 AM   #17
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
pb4uski's Avatar
 
Join Date: Nov 2010
Location: Vermont & Sarasota, FL
Posts: 16,458
I had not noticed the hyperlinks.... most useful part of the document IMO.

I did notice that they put in only partial information on their test scenario so readers could not try to replicate their results.
__________________
If something cannot endure laughter.... it cannot endure.
Patience is the art of concealing your impatience.
Slow and steady wins the race.
pb4uski is online now   Reply With Quote
Old 02-21-2016, 08:57 AM   #18
Thinks s/he gets paid by the post
sengsational's Avatar
 
Join Date: Oct 2010
Posts: 3,844
Quote:
Originally Posted by Meadbh View Post
The conclusion is that the design of publicly available calculators needs to be improved to avoid leading people astray. .
Quote:
Originally Posted by Midpack View Post
Some Company ABC retirement calculators are black boxes that don't reveal underlying assumptions and their purpose is to sell you Company ABC investments and/or financial advice - really, isn't that shocking!
Even if there's pages of assumptions associated with the calculator, if it just spits out "an answer", I'm going to be at least a bit skeptical. It would be better if I could download balances and flows year by year to see what it actually did. I'm sure there are calculators out there that have huge bugs that will never be found since we have only a black box / can only see the input and output, nothing in between.
__________________
sengsational is offline   Reply With Quote
Old 02-21-2016, 09:09 AM   #19
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
Midpack's Avatar
 
Join Date: Jan 2008
Location: Chicagoland
Posts: 11,978
Quote:
Originally Posted by sengsational View Post
Even if there's pages of assumptions associated with the calculator, if it just spits out "an answer", I'm going to be at least a bit skeptical. It would be better if I could download balances and flows year by year to see what it actually did. I'm sure there are calculators out there that have huge bugs that will never be found since we have only a black box / can only see the input and output, nothing in between.
FIRECALC will return year by year results FWIW. Though it's still difficult to recreate what happens in the black box.

There is no perfect retirement calculator or advisor, and never will be. You look at historical results, and make your own judgement re: how that will fare over the course of your retirement. It's up to you to factor in input variables, probabilities, etc. GIGO
__________________
No one agrees with other people's opinions; they merely agree with their own opinions -- expressed by somebody else. Sydney Tremayne
Retired Jun 2011 at age 57

Target AA: 60% equity funds / 35% bond funds / 5% cash
Target WR: Approx 2.5% Approx 20% SI (secure income, SS only)
Midpack is online now   Reply With Quote
Old 02-21-2016, 09:21 AM   #20
Thinks s/he gets paid by the post
sengsational's Avatar
 
Join Date: Oct 2010
Posts: 3,844
Quote:
Originally Posted by Midpack View Post
FIRECALC will return year by year results FWIW. Though it's still difficult to recreate what happens in the black box.
I think there's a spreadsheet 'out there' that has the same premise as FIRECalc and can be used to get at least roughly the same result. I've made a spreadsheet that validates i-orp output, given what it decided to do, flow-wise. So yeah, there are a few that lend themselves to sanity checking. I trust those more than a black box.
__________________

__________________
sengsational is offline   Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Study: 43% won't have enough in retirement dex FIRE and Money 28 06-27-2006 09:09 PM
Study: 43% won't have enough in retirement retire@40 FIRE and Money 2 06-07-2006 11:57 AM
Another Study on Early Retirement maddythebeagle Life after FIRE 17 10-21-2005 06:48 PM
New Study: What People Spend in Retirement REWahoo FIRE and Money 4 01-29-2005 04:29 PM
UK-Based study of early retirement finances TheFIREman Other topics 0 02-05-2004 04:10 AM

 

 
All times are GMT -6. The time now is 06:54 PM.
 
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2017, vBulletin Solutions, Inc.