using actual data or monte carlo sims-4% rule

[mathjak107]

dr wade pfau and david blanchett look at which is more accurate , using rolling actual time frames like firecalc or monte carlo simulations to determine allocations. interesting enough they feel the actual data when used is flawed becuause of to many overlapping time frames influenced by the same dates appearing multiple times skewing results ..

"The 4% rule did not fail in the historical period for stock allocations between 40% and 70%. More bond-heavy portfolios experienced much lower success rates, though, with a bonds-only portfolio succeeding just in 41% of the historical simulations. With Monte Carlo simulations based on the same historical data, retirees would be encouraged to hold some stocks, as success rates of over 90% are possible with stock allocations of only 20%. The highest success rates occurred in the range between 30% and 50% stocks."

Advisor Perspectives

 

[ERhoosier]

Basically agree with the main points of the article. More ammo to resist the aggressive FP's who seem to push for ever higher stock allocations (often via high cost funds &/or commissions). OTOH- Pfau's logic of rolling historical simulations being skewed by including multiples (single year retirements) of same long -term events could certainly apply to most recent 2-decade period of falling LT bond rates. Bond performance will be very different if we see a steady INcrease in rates over next 2 decades. This is where a Monte Carlo approach may be more predictive and relevant to next 20 yrs. Although I still do not understand how only 20% stock allocation could result in 90+% success of 4% WD's if we see rising rates
Bottom line:
Ya pays yer $$ & ya takes yer chances

 

[growing_older]

One critical issue is that real-life returns are linked to other real-life returns, possibly in ways we do not understand. I'm all for Monte Carlo approaches to get better assurance of robust results, but slight changes to Monte Carlo parameters can make big differences. At least with real-world results we KNOW that those results actually did happen. The danger we face is that it may be a skewed sample, but with Monte Carlo simulations we may be simulating scenarios that are actually outside what might ever occur (either positive or negative) and not know it.

 

[mathjak107]

Interesting how 20% equites and 70% equities had the same 93% success rate in the monte carlo.

makes you wonder about having high equity positions

 

[ERD50]

Interesting how 20% equites and 70% equities had the same 93% success rate in the monte carlo.

makes you wonder about having high equity positions

Makes me wonder about the Monte Carlo parameters.

For the record, as I've mentioned many times (but don't expect people to remember), I don't care for Monte Carlo for this case at all. There are interactions in the economy, and who knows what the computer is doing with this?

But it got me thinking - it might be interesting if the historic data that calculators like FIRECalc use was slightly randomized, to provide say, 100 similar versions of history. So we'd see what would happen when history rhymes. Questionable, but it might have value?

-ERD50

 

[FIRE'd@51]

Interesting how 20% equites and 70% equities had the same 93% success rate in the monte carlo.

makes you wonder about having high equity positions

Why? The 70% equity allocation should give you a significantly larger average portfolio value at the end of 30 years, probably on the order of 25-30%.

 

[FIRE'd@51]

probably on the order of 25-30%.

Actually , it's more like twice as large. I went back and looked at my Ibbotson spreadsheet and the real CPAR from 1926-2013 was 6.09% for 70% equities and 3.74% for 20% equities.

(1.0609/1.0374)^30 = 1.96

 

[mathjak107]

while we all believe the larger equity position do result in more wealth, unless the markets go straight up you may be winning by not losing.

don't forget with the data points used in multiple historical simulations what happens in one 30 year period drastically effects many other linked periods.

by equal weighting each 30 year time frame perhaps we are seeing unbiased results which much to our surprise may show very different results.

I can't really say in these new times who will be correct but it does get me thinking.

 

[photoguy]

Fortunately we don't have to pick just one withdrawal calculator and we can look at all of the results (and methodologies) before coming to our own conclusions. FIRECALC concerns me because of some of the issues that Pfau raises in his article (most specifically rolling periods, high current valuations, and selection bias from multiple countries). MC methods have their own drawbacks though and unless you carefully read the associated paper (and have experience to interpret it), you don't know what goes into the model.

 

[FIRE'd@51]

while we all believe the larger equity position do result in more wealth, unless the markets go straight up you may be winning by not losing.

IF you define risk as the portfolio failing to last 30 years, then, as you pointed out in post #4, the risk is the same for 20% equities and 70% equities; and 70% equities is clearly dominant because it has a higher expected return.

 

[mathjak107]

yes , when you are talking success rate that is what you are referring to, not going broke not the biggest pile at the end..

the traditional historical data used only 59 points with many data points sharing excessively good times so they carried over through as much as 3 -30 year time frames.

the monte carlo simulations ran 10,000 scenerios with most not sharing data points.

as far as success rates the 20% equities panning out quite successful does not surprise me.

if you remember michael kitces work where he defined just what it took for the 4% withdrawal rate to fail it was less than 2% real returns over the first 15 years of a 30 year time frame.

that is not much and using 20% equities certainly seems like a do under those conditions.

 

[nuke_diver]

I think MC sims are useful tools but because they are random you will almost always (with some reasonable assumptions) get failures. Using both historical data and MC is a good way to stress test your retirement plans...the historical shows you what has happened before and could happen again the MC shows you a randomized result....20 down years while we probably all agree would be highly improbably is possible in a MC sim.

 

[brewer12345]

If I were a researcher of dubious provenance and accomplishments, I would love Monte Carlo simulations. After all, its so much easier to make whatever point you like when you can make up your own data and do so in a manner that is so complex it is difficult for anyone to tell exactly what you faked...

 

[photoguy]

20 down years while we probably all agree would be highly improbably is possible in a MC sim.

Unless they've totally botched the MC sim, 20 consecutive down years should be so rare as to not occur in most sets of runs.

If I were a researcher of dubious provenance and accomplishments, I would love Monte Carlo simulations. After all, its so much easier to make whatever point you like when you can make up your own data and do so in a manner that is so complex it is difficult for anyone to tell exactly what you faked...

Yeah that's a huge drawback of MC methods for SWR studies -- it's going to take pages of description to describe how the MC sim is set up and probably almost nobody is going to double check it. They tend to have a s***load of parameters and it's not always clear how they are set or how much the author twiddled it before coming up with the value used. Not to mention that even if the logical design of the sim is "right", there may be bugs in the implementation.

In theory, peer review and replication is going to catch this but there are so few people working on SWR retirement simulations I suspect that serious errors could go undetected for a long time.

 

[walkinwood]

I'd trust MC simulations more if at least one of the runs was identical to the historical record.

Shouldn't that be a requirement to prove that their simulation is based on some semblance of reality?

 

[Texas Proud]

yes , when you are talking success rate that is what you are referring to, not going broke not the biggest pile at the end..

the traditional historical data used only 59 points with many data points sharing excessively good times so they carried over through as much as 3 -30 year time frames.

the monte carlo simulations ran 10,000 scenerios with most not sharing data points.

as far as success rates the 20% equities panning out quite successful does not surprise me.

if you remember michael kitces work where he defined just what it took for the 4% withdrawal rate to fail it was less than 2% real returns over the first 15 years of a 30 year time frame.

that is not much and using 20% equities certainly seems like a do under those conditions.

I was going to kinda say something similar.... the 'real' data has very few data points... I would prefer more data points to evaluate the possibilities.... with Monte Carlo, you can easily see a trend with it... of course there are some wild outliers, but the results seem to congregate around a mean...

So with real data, it might say you are 95%.... but you might not have good support for that %...

 

[mathjak107]

the real data includes zero time frames like we are in now which are low interest rates and high valuations. right off the bat our real historical data includes nothing we are seeing.

the monte carlo sims do. pfau and blanchett ran 10,000 sims.

pfau is also correct that the middle years have over lapping data points that show up as many as 30x . the 1955 to 1984 time frame fits that description.

it was also a time of one of the worst bond performance periods we had so that means bonds success rate alone would be skewed and show lots of failure since those years appeared in so many other rolling periods.

personally i think the actual historical will not repeat in the same order and so many time frames would have shown different results the 2nd time around if the data was not repeating from previous.

the time frame from 1987 to 2003 was the greatest bull run ,17 years at almost a 14.00% average return. real data wise how many time frames will that skew with an event that may never happen again that way? a lot of upcoming time frames will be running on those numbers and showing great success rates from equities .

 

[photoguy]

I'd trust MC simulations more if at least one of the runs was identical to the historical record.

Shouldn't that be a requirement to prove that their simulation is based on some semblance of reality?

Yes. The authors of the sim should run it under historical assumptions which would hopefully give results in roughly the same ballpark as Firecalc/Trinity/Bengen. Even if they don't publish this result they should still be doing it as part of development and testing.

At least in some papers Pfau has run multiple scenarios with one corresponding to US historical data. It's sobering though to see that the mean return for the historical sim is a huge amount higher then what they call the "likely" case for future returns (if I remember correctly its like 8% vs 5% for equities).

 

[brewer12345]

Yes. The authors of the sim should run it under historical assumptions which would hopefully give results in roughly the same ballpark as Firecalc/Trinity/Bengen. Even if they don't publish this result they should still be doing it as part of development and testing.

At least in some papers Pfau has run multiple scenarios with one corresponding to US historical data. It's sobering though to see that the mean return for the historical sim is a huge amount higher then what they call the "likely" case for future returns (if I remember correctly its like 8% vs 5% for equities).

But all those alternate realities are based on the Evil Dr. Pfau's favorite tool: made up numbers. So you don't like using the historical record and would rather base your plans on numbers Pfau essentially pulled out of his ass? Am I the only one who thinks this is problematic?

 

[jimbee]

But all those alternate realities are based on the Evil Dr. Pfau's favorite tool: made up numbers. So you don't like using the historical record and would rather base your plans on numbers Pfau essentially pulled out of his ass? Am I the only one who thinks this is problematic?

No, you're not the only one. I think using Monte Carlo models for financial forecasting is one step away from a Ouja board. You can make lots of interesting looking graphs and charts, but the input numbers are basically plucked from thin air.

 

[ERD50]

No, you're not the only one. I think using Monte Carlo models for financial forecasting is one step away from a Ouja board. You can make lots of interesting looking graphs and charts, but the input numbers are basically plucked from thin air.

+2, as I've said many times before.

I have to shake my head when I see comments about adjusting the parameters until you the similar results to historic reports. What's the point then?

To be fair, I think a recent comment meant to use historical data to check the math functions for the MC calculation is different. But I still don;t see the point of MC for this use. It makes sense for many other types of analysis, but I sometimes wonder about that...

Our design engineers used MC to simulate different component variations, all within tolerance, but to determine if the circuit will still meet spec with random variations within those tolerances. But components are purchased in bulk, and some of them may have a 10% tolerance, but within a single batch might run very, very close to each other. I'm not sure they modeled that well.

-ERD50

 

[haha]

But all those alternate realities are based on the Evil Dr. Pfau's favorite tool: made up numbers. So you don't like using the historical record and would rather base your plans on numbers Pfau essentially pulled out of his ass? Am I the only one who thinks this is problematic?

I think it all has problems; not sure one is better than the other. I pitch my tent on what Buffett used to call look-through earnings.

Unless you are going to liquidate you investments and just run down the cash, the quoted price is of 2ndary importance. What counts is firm level real earnings and cash generation.

Ha

 

[brewer12345]

I think it all has problems; not sure one is better than the other. I pitch my tent on what Buffett used to call look-through earnings.

Unless you are going to liquidate you investments and just run down the cash, the quoted price is of 2ndary importance. What counts is firm level real earnings and cash generation.

Ha

Yep, it all has problems. That is life. At least the historical record is verifiable.

I guess I take a more expansive view. The earnings a firm generates can change dramatically in a short time, as the last several years have shown. The historical record may or may not apply in a world where million to one chances seem to pop up nine times out of ten. Dr. Pfau may be pulling numbers out of his nether regions. The Islamic State may prove an excuse for yet another round of overpriced, ill-advised war and a removal of what few remaining civil rights we have (I think this is the most likely of these bets). We live in an uncertain world. You pays your money and you takes your chances. I personally have a very long timeline, so I expect to take risk, take my lumps and buttress my portfolio with periodic earned income.

 

[Fred123]

This topic has come up before, and I'm always surprised at the aversion some people here have to MC analysis and the overconfidence they have in FC (which relies on perhaps 5 independent samples if you are looking at 30 year outcomes). Some of this seems to be based on a misunderstanding on how MC works if historical data is used to model the distribution. For example, there aren't "a s***load of parameters" used, there are two (mean and variance) for normal or log-normal distributions and maybe one or two others if a "fat-tailed" distribution is used. There are no parameters used at all if bootstrap sampling is used.

But, more importantly, what are the alternatives? As a famous statistician once said: "all models are wrong, but some are useful."

 

[Options]

Otar basically blows MC's out of the water as well. What to do? You could try using both MC's and historical data. Be conservative in all assumptions as well.