Monte Carlo Simulations-now recognized as understating risk

[ejman]

Quote:

Originally Posted by samclem I think the article overstates the significance of adding more simulation runs. That's not an important source of the expected vs real-world discrepency we are all seeing today. If the assumed variability in asset class returns represented what we have seen in the real world, and if the correlations between asset class returns were the same as what have actually occurred, then it wouldn't matter much if you ran 5,000 iterations or a million, the failures would show up. FIRECalc ain't perfect, but it does at least include the worst-seen-to-date data. As the data from the present market stumble gets included, you'll have some more fatness in the tails. And, at that point, the relatively limited number of data years in FIRECalc will be an asset, serving to assure the "stinkers" get appropriate weight. This isn't a once-in-a-million event we're experiencing now.

Agreed. An additional point to consider re FIRECalc is that the actual investment return data for the last century includes the time period when the US turned into the preeminent world power with an increase in wealth and productivity that may be difficult to match going forward. I wonder what the results of a FIRECalc type analysis would be if instead of using US data one were to use say Germany's or Japan or heaven forbid Argentina!

[charlie]

"The best laid plans of mice and men ..... etc." Always have a Plan B
in your hip pocket.

Cheers,

charlie

[deserat]

"Critics emphasize that the problem isn't Monte Carlo itself, but the assumptions that go into it."

The most important sentence in the article - hello! Anybody who's had any math/engineering background know the assumptions make/break you. Expecting a model with assumptions to get even close to real life close to 100% of the time means one is quite naive. Bernstein's right - add a little extra padding - for him it's 20%......

Just because you've run the equation/model millions of times doesn't mean it predicts the true outcome - gotta have the right equation......GIGO

[Midpack]

Quote:

Originally Posted by deserat "Critics emphasize that the problem isn't Monte Carlo itself, but the assumptions that go into it." The most important sentence in the article - hello! Anybody who's had any math/engineering background know the assumptions make/break you. Expecting a model with assumptions to get even close to real life close to 100% of the time means one is quite naive. Bernstein's right - add a little extra padding - for him it's 20%...... Just because you've run the equation/model millions of times doesn't mean it predicts the true outcome - gotta have the right equation......GIGO

Life is uncertain, it's curious that anyone assumes otherwise in retirement (see virtually all online retirement calculators). Financial plans are no different, and no amount of modeling can change that. However, adding MC is still far, far better than the constant return/inflation & fixed longevity models that many people use if for no other purpose than to imprint the concept of uncertainty and planning for same for those who haven't thought through it. I have read this The Retirement Calculator from Hell, Part III several times, and keep it in mind always. I think Dr. Bernstein's whole 5-part Retirement Calculator From Hell series is worthwhile reading for everyone planning their own retirement (the non-SIRE crowd). 95% probability is more secure than 80% - it's just not actually 95%...

Quote:

A wildly optimistic historian might give us another few centuries of economic, political, and military continuity. Back-of-the-envelope, that’s about an 80% survival rate over the next 40 years. Thus, any estimate of long-term financial success greater than about 80% is meaningless.

[Independent]

Sam questions this statement. I do, too:

Quote:

Some industry participants also are trying to set standards that could help Monte Carlo tools more accurately capture extreme market events. The Retirement Income Industry Association in 2007 issued a set of principles noting that the calculators should run a large number of scenarios. The ideal models run tens of thousands or hundreds of thousands of scenarios, which help gauge extreme events at the tail end of the distribution, observers said. Yet some tools run only 1,000 scenarios or just several hundred.

For retirement planning, lots of people accept 95% probability from the model. AFAIK, for a fixed set of assumptions, the 95% SWR is effectively the same whether you run 1,000 or 100,000 scenarios.

You're better off thinking about things outside the model then worrying about running a lot more scenarios.

[Dennis]

Well, I looked at alot of these Self Proclaimed Guru's Plans and methods and systems, prior to about 5 yrs before retiring... and then I just used my own Simple Minded Common sense...

Example: Bernstiens Retirement from Hell..."To make $100k yr at a 4% SWR you neng ave of $1. 769k?"
I don't know where he gets his #'s, but I come up with $250k and After paying 25% in Fed and state taxes, I'd have 3% net to spend. and adding another 4% or 3% net after taxes left in for inflation/buying power.. tells me I need a min. 8% apy on my $. and it's reasonable to make 10% apy on a 60/40 port? I don';t know about that either, a 60/40 port of indexes, Ending in 07' only had an ave of about 7.4% apy for the previous 10 yrs, add 08' and it's now the past 10 yrs ending 08' of only about 3.3% apy..

This , among other reasons moved me to get out of Indexes and get into Active Balanced Funds.. way back in 98/99', that had and still have the best chance of meeting and succeeding that 8% apy I'm looking for..and even after 08', the 4 BF's I have are still in the 9% apy range for the past 9 yrs now.. and I added an extra 20% to what everyone and Myself figured I would need to have ...thus worked an extra 4 yrs...

and being a person who believes you need a Speicalist for everything Now-a-days and Most of Us Amatures Haven't Got a Chance doing out own Investing..and we Need a Pro' to Beat a Pro'...and one with a proven record and not therories or Charts...with Real $..( most Advisors Won't share what their Past 3,5 & 10 yr Recommended ports did to you )

and everytime ( about every 6mos) I go see one of these Financial Advisors or A Firm offerring to do a Port. review to get my $? After they give me their recomendations for a conservative to Moderate Port? I pull out mine and none have yet to get close, let alone beat them and they critize them and hate them...

Which tells me, I'm on the right course...LOL

But, Like the Guy who Jumped off a 40 story building , said at the 30th floor?
Well, So Far, So good...
;>)

[samclem]

Quote:

Originally Posted by Dennis Example: Bernstiens Retirement from Hell..."To make $100k yr at a 4% SWR you neng ave of $1. 769k?"

He didn't say that.
(From "Retirement Calculator From Hell-Part III")

Quote:

For centuries, investors used the amortization algorithm—the same formula used to calculate mortgages. Let’s say that you plan a 30-year retirement, estimate a 4% real return, and need $100,000 in annual income. Toss these figures into your trusty retirement calculator, and hey presto, out pops a required nest egg of $1,729,203.

Note that the 4% is real return (i.e. after inflation), and that every last penny is spent by the end of 30 years. Also, of course, he goes on to say how silly these straight-line calculations are, proposes MC as something better, then notes the limitations of any quantitative modelling.

[FinanceDude]

Monte Carlo has always been just "ok". Most if not all financial theorists have never seen the "prefect storm" of equities crashing, bonds getting haircuts, and historically low interest rates all in the same year.

I imagine it has Markowitz and all his buddied reconsidering their theories of MPT and such........

I never hear too much about beta. A little about alpha, but not much. I guess those terms are unsexy......

[Gearhead Jim]

In aeordynamics, designers often use Computational Fluid Dynamics, abbreviated "CFD", to predict how airflow will work.
Some of them sing the praises of CFD.
Others, who've had problems with the modeling, say that CFD stands for "Can't F*****g Decide"
Models are uh, less than perfect.

[NW-Bound]

Being an EE, I do not use CFD but can relate to what G-J described. In a previous project, I worked with a highly accurate electronic device that would be affected by the flexure of the mechanical structure that the device was mounted on. So, the mechanical department ran their NASTRAN model (a finite-element analysis method) to predict the flexure for me to compensate. Their result turned out all wrong! It did not even have the right sign; meaning they could not tell if it was coming or going, for crying out loud! So, we just measure the actual flexure across several prototypes, and use the average of the empirical data.

All the above does not mean that I do not use FireCalc. However, I only use it as a guide, as its author has said. "Don't measure with a caliper what you are going to cut with an axe", or something like that.

I believe in Uncle Mick's practical method. In lean years, tighten your belt. In good years, splurge. You can also splurge when you get older, as you cannot take it with you. Heh heh heh...

[jdw_fire]

Quote:

Originally Posted by NW-Bound Being an EE, I do not use CFD but can relate to what G-J described. In a previous project, I worked with a highly accurate electronic device that would be affected by the flexure of the mechanical structure that the device was mounted on. So, the mechanical department ran their NASTRAN model (a finite-element analysis method) to predict the flexure for me to compensate. Their result turned out all wrong! It did not even have the right sign; meaning they could not tell if it was coming or going, for crying out loud! So, we just measure the actual flexure across several prototypes, and use the average of the empirical data. All the above does not mean that I do not use FireCalc. However, I only use it as a guide, as its author has said. "Don't measure with a caliper what you are going to cut with an axe", or something like that. I believe in Uncle Mick's practical method. In lean years, tighten your belt. In good years, splurge. You can also splurge when you get older, as you cannot take it with you. Heh heh heh...

FWIW FireCalc is more like what you did that worked, i.e. measure what actually happened, and then use that data to set a boundry.

[Ed_The_Gypsy]

Quote:

"All models are wrong. Some models are useful." - - George E.P. Box

I love this, too.

I once worked for a guy who thought that models should simulate reality accurately in all respects. (This is f or a chemical plant simulator, but it applies to every kind of simulation.) Guy was a nut. Did not understand the utility and limits of simulations.

[Htown Harry]

Here's an article posted today relevant to this thread:

Retirement tools often underestimate risk Robert Powell - MarketWatch

The article summarizes a paper by Richard Fullmer, who is mentioned in the WSJ article in the OP's post.

http://www.plansponsor.com/uploadfil...rment-Risk.pdf

From the Marketwatch article:

"Models that measure only the probability of failure ignore one side of the risk equation completely."

Fullmer isn't suggesting that Monte Carlo be scrapped. Simulation models actually lend themselves very well to proper risk measurement but only if they measure both the probability of failure and the magnitude of the failure cases that occur. Right now, however, the models use this calculation: The magnitude is the total amount of desired spending and bequeathing (how much money is left to heirs and others after you die) that does not occur because the portfolio ran out of money -- often referred to as "shortfall." In other words, the existing tools use this model: Shortfall risk equals probability of shortfall.

A better risk measure, however, would be this: Shortfall risk equals probability of shortfall times the magnitude of shortfall.

[haha]

If a carpenter understood his tools as poorly as the average retiree or retirement planner understands his, many houses would fall down.

What this author seems to be trying to do is differentiate between risk and expectation. Any gambler who can even break even understands this.

But many if not most retirement board and FA discussions are glib recitals that often miss the point. Command of the lingo and a dubious precision don't equal wisdom. Like Fullmer says, many of us may be accepting too much chance of catastrophe. A high equity allocation does apparently give a lower probability of failure over longer time periods, but the failures can be real doozies.

A retirement academic like Moshe Milevsky writes clearly about this issue.

I think that the past 18 months may have reinforced this damn the torpedoes approach as we all got scared strongly, but then quickly let off the hook. This is a classical desensitization treatment. "Hey. I went over the falls but here I am."

It would be hard not to conclude that Little Ben isn't correct when he suggests that given enough money the Fed can fix anything.

But next time could easily be different.

I believe that the truth is that it is very hard to live for a long period of time without wage or active business or professional income, or a secure stream of inflation adjusted retirement payments. This may be unpleasant, and it may turn out to be false, but I would advise a high burden of proof before rejecting it.

Ha

[RonBoyd]

Quote:

Originally Posted by haha I believe that the truth is that it is very hard to live for a long period of time without wage or active business or professional income, or a secure stream of inflation adjusted retirement payments. This may be unpleasant, and it may turn out to be false, but I would advise a high burden of proof before rejecting it.

What he said. A point seemingly missed by many.

[walkinwood]

There are no mathematical or physical laws that underlie ANY market model or simulation, so there's always the chance that these simulations understate risks.

Any retirement plan that does not include a behavioral component is flawed. By that I mean, a component that specifies behavior changes in response to financial conditions.

Even people living on COLA pensions need to be ready to adjust to financial conditions. How are the Iraqis on government pensions doing? or the Russians, or the Argentinians?

[rmark]

'I wonder what the results of a FIRECalc type analysis would be if instead of using US data one were to use say Germany's or Japan or heaven forbid Argentina!'

Actually, I took the indiviual countries data 1900-2000 from 'Triumph of the Optimists' and ran it through the moneychimp.com monte carlo demonstrator using a 60% stock/40% bond portfolio and an initial 4% withdrawal from a $1,000,000 over 25 years.
Germany (ex-weimar republic) had a 40% success rate, Japan 42%, but no Argentina. Italy was the low at 35%. The US success rate was 83% , probably due to the slightly lower returns the book lists.

[ziggy29]

In reality, Monte Carlo can only "estimate" or "simulate" risks that have already occurred. If you understand it in THAT context, with the understanding that the future may not resemble the past, it is likely prudent to add a "margin of safety" to their estimates to account for the potential future scenarios that Monte Carlo can't predict.

[Bikerdude]

Financial simulations are like the old "double slit experiment". They act one way when you run them and another when you actually try to execute them in real life while you watch.

YouTube - Dr Quantum - Double Slit Experiment

[unclemick]

Sooo - like er ah 'mentioned' over at the Bogleheads - how do you say Pssst- Dividends in Norwegian? Better than beer in Austrialian.

I could have said pssst Wellesley. But this forum has a lock on the good stuff.

In hard times nailing down some income is sometimes better than over trying to catch inflation.

heh heh heh - remember what S.I. Hayakawa said math models are not always stocks that pay dividends. ok ok so he said something else. .