Monte Carlo Simulations-now recognized as understating risk

[unclemick]

Long long ago - in another time - tears and rage from the number cats when the models failed wind tunnel/re-entry simulation/shake rattle and roll sims.

Obviously us/we make em and break em empirical weenies made the model wrong or screwed up the test parameters - cause THEIR numbers did not lie!



Loved models - both physical and numerical.

heh heh heh - now in ER chickenheartedness demands some SEC yield - just in case.

 

[Gotadimple]

Loved models - both physical and numerical.

heh heh heh - now in ER chickenheartedness demands some SEC yield - just in case.

Uncle Mick,
You always provide the right kind of encouragement. Models are just models, and as much as we'd like to make them so they could predict a result, they are still just models without superior knowledge of probabilities.

So to continue to provide constructive criticism on how to improve the models strikes me as analysis paralysis.

Better to remember that SEC yield thingie (or the other Psst doohickey you occasionally mention).

-- Rita

P.S. Don't own the doohickey, but am beginning to appreciate the advantage of the thingie.

 

[charlie]

"The best laid plans of mice and men ..... etc." Always have a Plan B
in your hip pocket.

Cheers,

charlie

 

[FinanceDude]

Monte Carlo has always been just "ok". Most if not all financial theorists have never seen the "prefect storm" of equities crashing, bonds getting haircuts, and historically low interest rates all in the same year.

I imagine it has Markowitz and all his buddied reconsidering their theories of MPT and such........

I never hear too much about beta. A little about alpha, but not much. I guess those terms are unsexy......

 

[Gearhead Jim]

In aeordynamics, designers often use Computational Fluid Dynamics, abbreviated "CFD", to predict how airflow will work.
Some of them sing the praises of CFD.
Others, who've had problems with the modeling, say that CFD stands for "Can't F*****g Decide"
Models are uh, less than perfect.

 

[NW-Bound]

Being an EE, I do not use CFD but can relate to what G-J described. In a previous project, I worked with a highly accurate electronic device that would be affected by the flexure of the mechanical structure that the device was mounted on. So, the mechanical department ran their NASTRAN model (a finite-element analysis method) to predict the flexure for me to compensate. Their result turned out all wrong! It did not even have the right sign; meaning they could not tell if it was coming or going, for crying out loud! So, we just measure the actual flexure across several prototypes, and use the average of the empirical data.

All the above does not mean that I do not use FireCalc. However, I only use it as a guide, as its author has said. "Don't measure with a caliper what you are going to cut with an axe", or something like that.

I believe in Uncle Mick's practical method. In lean years, tighten your belt. In good years, splurge. You can also splurge when you get older, as you cannot take it with you. Heh heh heh...

 

[jdw_fire]

Being an EE, I do not use CFD but can relate to what G-J described. In a previous project, I worked with a highly accurate electronic device that would be affected by the flexure of the mechanical structure that the device was mounted on. So, the mechanical department ran their NASTRAN model (a finite-element analysis method) to predict the flexure for me to compensate. Their result turned out all wrong! It did not even have the right sign; meaning they could not tell if it was coming or going, for crying out loud! So, we just measure the actual flexure across several prototypes, and use the average of the empirical data.

All the above does not mean that I do not use FireCalc. However, I only use it as a guide, as its author has said. "Don't measure with a caliper what you are going to cut with an axe", or something like that.

I believe in Uncle Mick's practical method. In lean years, tighten your belt. In good years, splurge. You can also splurge when you get older, as you cannot take it with you. Heh heh heh...

FWIW FireCalc is more like what you did that worked, i.e. measure what actually happened, and then use that data to set a boundry.

 

[Ed_The_Gypsy]

"All models are wrong. Some models are useful." - - George E.P. Box

I love this, too.

I once worked for a guy who thought that models should simulate reality accurately in all respects. (This is f or a chemical plant simulator, but it applies to every kind of simulation.) Guy was a nut. Did not understand the utility and limits of simulations.

 

[Htown Harry]

Here's an article posted today relevant to this thread:

Retirement tools often underestimate risk Robert Powell - MarketWatch

The article summarizes a paper by Richard Fullmer, who is mentioned in the WSJ article in the OP's post.

http://www.plansponsor.com/uploadfiles/Russell_Mismeasurment-Risk.pdf

From the Marketwatch article:

"Models that measure only the probability of failure ignore one side of the risk equation completely."

Fullmer isn't suggesting that Monte Carlo be scrapped. Simulation models actually lend themselves very well to proper risk measurement but only if they measure both the probability of failure and the magnitude of the failure cases that occur. Right now, however, the models use this calculation: The magnitude is the total amount of desired spending and bequeathing (how much money is left to heirs and others after you die) that does not occur because the portfolio ran out of money -- often referred to as "shortfall." In other words, the existing tools use this model: Shortfall risk equals probability of shortfall.

A better risk measure, however, would be this: Shortfall risk equals probability of shortfall times the magnitude of shortfall.​

 

[haha]

If a carpenter understood his tools as poorly as the average retiree or retirement planner understands his, many houses would fall down.

What this author seems to be trying to do is differentiate between risk and expectation. Any gambler who can even break even understands this.

But many if not most retirement board and FA discussions are glib recitals that often miss the point. Command of the lingo and a dubious precision don't equal wisdom. Like Fullmer says, many of us may be accepting too much chance of catastrophe. A high equity allocation does apparently give a lower probability of failure over longer time periods, but the failures can be real doozies.

A retirement academic like Moshe Milevsky writes clearly about this issue.

I think that the past 18 months may have reinforced this damn the torpedoes approach as we all got scared strongly, but then quickly let off the hook. This is a classical desensitization treatment. "Hey. I went over the falls but here I am."

It would be hard not to conclude that Little Ben isn't correct when he suggests that given enough money the Fed can fix anything.

But next time could easily be different.

I believe that the truth is that it is very hard to live for a long period of time without wage or active business or professional income, or a secure stream of inflation adjusted retirement payments. This may be unpleasant, and it may turn out to be false, but I would advise a high burden of proof before rejecting it.

Ha

 

[walkinwood]

There are no mathematical or physical laws that underlie ANY market model or simulation, so there's always the chance that these simulations understate risks.

Any retirement plan that does not include a behavioral component is flawed. By that I mean, a component that specifies behavior changes in response to financial conditions.

Even people living on COLA pensions need to be ready to adjust to financial conditions. How are the Iraqis on government pensions doing? or the Russians, or the Argentinians?

 

[RonBoyd]

I believe that the truth is that it is very hard to live for a long period of time without wage or active business or professional income, or a secure stream of inflation adjusted retirement payments. This may be unpleasant, and it may turn out to be false, but I would advise a high burden of proof before rejecting it.

What he said. A point seemingly missed by many.

 

[rmark]

'I wonder what the results of a FIRECalc type analysis would be if instead of using US data one were to use say Germany's or Japan or heaven forbid Argentina!'

Actually, I took the indiviual countries data 1900-2000 from 'Triumph of the Optimists' and ran it through the moneychimp.com monte carlo demonstrator using a 60% stock/40% bond portfolio and an initial 4% withdrawal from a $1,000,000 over 25 years.
Germany (ex-weimar republic) had a 40% success rate, Japan 42%, but no Argentina. Italy was the low at 35%. The US success rate was 83% , probably due to the slightly lower returns the book lists.

 

[ziggy29]

In reality, Monte Carlo can only "estimate" or "simulate" risks that have already occurred. If you understand it in THAT context, with the understanding that the future may not resemble the past, it is likely prudent to add a "margin of safety" to their estimates to account for the potential future scenarios that Monte Carlo can't predict.

 

[Bikerdude]

Financial simulations are like the old "double slit experiment". They act one way when you run them and another when you actually try to execute them in real life while you watch.

YouTube - Dr Quantum - Double Slit Experiment

 

[unclemick]

Sooo - like er ah 'mentioned' over at the Bogleheads - how do you say Pssst- Dividends in Norwegian? Better than beer in Austrialian.

I could have said pssst Wellesley. But this forum has a lock on the good stuff.

In hard times nailing down some income is sometimes better than over trying to catch inflation.

heh heh heh - remember what S.I. Hayakawa said math models are not always stocks that pay dividends. ok ok so he said something else. .

 

[Gone4Good]

I guess that's it for the Early Retirement Forums.

Please turn out the lights if you're the last one to leave.

 

[Maurice]

In reality, Monte Carlo can only "estimate" or "simulate" risks that have already occurred. If you understand it in THAT context, with the understanding that the future may not resemble the past, it is likely prudent to add a "margin of safety" to their estimates to account for the potential future scenarios that Monte Carlo can't predict.

Not really, MC analyses use a stochastic model that doesn't bear any relation to historical data. Depending on the model used, you could simulate a world with much fatter tails than anything we've experienced, or vice versa.

 

[Bikerdude]

Not really, MC analyses use a stochastic model that doesn't bear any relation to historical data. Depending on the model used, you could simulate a world with much fatter tails than anything we've experienced, or vice versa.

That was my understanding of MC simulation. This is why they say 90% success rate is pretty good on MC because of all the "end of world" scenario's included, that while possible, are not probable.

 

[samclem]

That was my understanding of MC simulation. This is why they say 90% success rate is pretty good on MC because of all the "end of world" scenario's included, that while possible, are not probable.

You don't really know if 90% success is good or not unless you know about the underlying data being used. It would be easy (though irresponsible) to have an assumed data set that was more optimistic (i.e. fewer very bad years) than those already seen in the real world, in which case even 100% success could be produced by a really bad portfolio.

It's all about the "fattness" of the tails of the distribution.

 

[ziggy29]

Not really, MC analyses use a stochastic model that doesn't bear any relation to historical data. Depending on the model used, you could simulate a world with much fatter tails than anything we've experienced, or vice versa.

This can be true but I think it muddies the waters even more. How do we know how to assign the probability of something that hasn't happened in the modern history of the developed markets?

I think trying to use MC for anything more than seeing how you would fare under the universe of actual results is about the best we can hope for, and then add a bit of a margin of safety that might help increase survivability through unprecedented future market environments.

 

[Bikerdude]

You don't really know if 90% success is good or not unless you know about the underlying data being used. It would be easy (though irresponsible) to have an assumed data set that was more optimistic (i.e. fewer very bad years) than those already seen in the real world, in which case even 100% success could be produced by a really bad portfolio.

It's all about the "fattness" of the tails of the distribution.

That's the general complaint about MC simulations.