Join Early Retirement Today
Reply
 
Thread Tools Display Modes
Old 09-10-2014, 08:35 AM   #21
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
 
Join Date: Sep 2005
Location: Northern IL
Posts: 26,821
Quote:
Originally Posted by jimbee View Post
No, you're not the only one. I think using Monte Carlo models for financial forecasting is one step away from a Ouja board. You can make lots of interesting looking graphs and charts, but the input numbers are basically plucked from thin air.
+2, as I've said many times before.

I have to shake my head when I see comments about adjusting the parameters until you the similar results to historic reports. What's the point then?

To be fair, I think a recent comment meant to use historical data to check the math functions for the MC calculation is different. But I still don;t see the point of MC for this use. It makes sense for many other types of analysis, but I sometimes wonder about that...

Our design engineers used MC to simulate different component variations, all within tolerance, but to determine if the circuit will still meet spec with random variations within those tolerances. But components are purchased in bulk, and some of them may have a 10% tolerance, but within a single batch might run very, very close to each other. I'm not sure they modeled that well.

-ERD50
ERD50 is offline   Reply With Quote
Join the #1 Early Retirement and Financial Independence Forum Today - It's Totally Free!

Are you planning to be financially independent as early as possible so you can live life on your own terms? Discuss successful investing strategies, asset allocation models, tax strategies and other related topics in our online forum community. Our members range from young folks just starting their journey to financial independence, military retirees and even multimillionaires. No matter where you fit in you'll find that Early-Retirement.org is a great community to join. Best of all it's totally FREE!

You are currently viewing our boards as a guest so you have limited access to our community. Please take the time to register and you will gain a lot of great new features including; the ability to participate in discussions, network with our members, see fewer ads, upload photographs, create a retirement blog, send private messages and so much, much more!

Old 09-10-2014, 10:40 AM   #22
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
haha's Avatar
 
Join Date: Apr 2003
Location: Hooverville
Posts: 22,983
Quote:
Originally Posted by brewer12345 View Post
But all those alternate realities are based on the Evil Dr. Pfau's favorite tool: made up numbers. So you don't like using the historical record and would rather base your plans on numbers Pfau essentially pulled out of his ass? Am I the only one who thinks this is problematic?
I think it all has problems; not sure one is better than the other. I pitch my tent on what Buffett used to call look-through earnings.

Unless you are going to liquidate you investments and just run down the cash, the quoted price is of 2ndary importance. What counts is firm level real earnings and cash generation.

Ha
__________________
"As a general rule, the more dangerous or inappropriate a conversation, the more interesting it is."-Scott Adams
haha is offline   Reply With Quote
Old 09-10-2014, 11:15 AM   #23
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
 
Join Date: Mar 2003
Posts: 18,085
Quote:
Originally Posted by haha View Post
I think it all has problems; not sure one is better than the other. I pitch my tent on what Buffett used to call look-through earnings.

Unless you are going to liquidate you investments and just run down the cash, the quoted price is of 2ndary importance. What counts is firm level real earnings and cash generation.

Ha
Yep, it all has problems. That is life. At least the historical record is verifiable.

I guess I take a more expansive view. The earnings a firm generates can change dramatically in a short time, as the last several years have shown. The historical record may or may not apply in a world where million to one chances seem to pop up nine times out of ten. Dr. Pfau may be pulling numbers out of his nether regions. The Islamic State may prove an excuse for yet another round of overpriced, ill-advised war and a removal of what few remaining civil rights we have (I think this is the most likely of these bets). We live in an uncertain world. You pays your money and you takes your chances. I personally have a very long timeline, so I expect to take risk, take my lumps and buttress my portfolio with periodic earned income.
__________________
"All animals are equal, but some animals are more equal than others."

- George Orwell

Ezekiel 23:20
brewer12345 is offline   Reply With Quote
Old 09-10-2014, 12:16 PM   #24
Recycles dryer sheets
 
Join Date: May 2013
Posts: 127
This topic has come up before, and I'm always surprised at the aversion some people here have to MC analysis and the overconfidence they have in FC (which relies on perhaps 5 independent samples if you are looking at 30 year outcomes). Some of this seems to be based on a misunderstanding on how MC works if historical data is used to model the distribution. For example, there aren't "a s***load of parameters" used, there are two (mean and variance) for normal or log-normal distributions and maybe one or two others if a "fat-tailed" distribution is used. There are no parameters used at all if bootstrap sampling is used.

But, more importantly, what are the alternatives? As a famous statistician once said: "all models are wrong, but some are useful."
Fred123 is offline   Reply With Quote
Old 09-10-2014, 12:40 PM   #25
Thinks s/he gets paid by the post
 
Join Date: Jan 2008
Posts: 1,495
Otar basically blows MC's out of the water as well. What to do? You could try using both MC's and historical data. Be conservative in all assumptions as well.
Options is offline   Reply With Quote
Old 09-10-2014, 02:05 PM   #26
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
 
Join Date: Sep 2005
Location: Northern IL
Posts: 26,821
Quote:
Originally Posted by Fred123 View Post
This topic has come up before, and I'm always surprised at the aversion some people here have to MC analysis and the overconfidence they have in FC (which relies on perhaps 5 independent samples if you are looking at 30 year outcomes). Some of this seems to be based on a misunderstanding on how MC works if historical data is used to model the distribution. For example, there aren't "a s***load of parameters" used, there are two (mean and variance) for normal or log-normal distributions and maybe one or two others if a "fat-tailed" distribution is used. There are no parameters used at all if bootstrap sampling is used.

But, more importantly, what are the alternatives? As a famous statistician once said: "all models are wrong, but some are useful."
Who says anyone has 'overconfidence in FC'? This was recently covered in the 'pet peeve' thread - please don't put words in other posters's posts.

From my perspective, FC reports history. I am reasonably confident that it does this reasonably accurately (there are some questions about the bond re-balancing algorithm). What you decide to do based on that report is a personal choice. But I don't think it infers an 'overconfidence' in it. It shows ~ 100 different lines - which one am I supposedly 'confident' in? And I understand the future could be worse than the worst of the past. But I think it's a reasonable data-point.

You say that MC uses only two parameters? But on how many variables? FC is looking at market returns, fixed returns, and inflation. I don't believe those are totally independent, so how does MC handle any correlations (positive or negative)? Without a full understanding of that, I don't know what to think about any MC run.

FWIW, I approach the historical calculators conservatively. I plan for the outside chance one of us lives to 100 (beyond that is diminishing changes anyhow). I choose 100% success rates. Then I throw in a little buffer. How does that make me 'overconfident'?


edit/add: I agree with you that there are really only a few economic cycles represented in this history. But what else can we do? You need to start somewhere.

-ERD50
ERD50 is offline   Reply With Quote
Old 09-10-2014, 03:55 PM   #27
Moderator Emeritus
M Paquette's Avatar
 
Join Date: Oct 2007
Location: Portland
Posts: 4,946
I love Monte Carlo simulations. Especially when I can buttonhole the designer and ask questions.

"I see you used a Gaussian random number generator. Does the output actually resemble the market behavior of that asset? How much divergence? I don't see where you allow for cross-correlated behavior for these different asset types? Does the histogram of first and second derivatives with respect to time resemble the behavior of the asset in the real world? How closely?"

The answers usually resemble the series "Um. Uh... I don't know. What? The who? Uh..."
M Paquette is offline   Reply With Quote
Old 09-10-2014, 05:53 PM   #28
Recycles dryer sheets
 
Join Date: May 2013
Posts: 127
Quote:
Originally Posted by ERD50 View Post
Who says anyone has 'overconfidence in FC'? This was recently covered in the 'pet peeve' thread - please don't put words in other posters's posts.

From my perspective, FC reports history. I am reasonably confident that it does this reasonably accurately (there are some questions about the bond re-balancing algorithm). What you decide to do based on that report is a personal choice. But I don't think it infers an 'overconfidence' in it. It shows ~ 100 different lines - which one am I supposedly 'confident' in? And I understand the future could be worse than the worst of the past. But I think it's a reasonable data-point.

You say that MC uses only two parameters? But on how many variables? FC is looking at market returns, fixed returns, and inflation. I don't believe those are totally independent, so how does MC handle any correlations (positive or negative)? Without a full understanding of that, I don't know what to think about any MC run.

FWIW, I approach the historical calculators conservatively. I plan for the outside chance one of us lives to 100 (beyond that is diminishing changes anyhow). I choose 100% success rates. Then I throw in a little buffer. How does that make me 'overconfident'?


edit/add: I agree with you that there are really only a few economic cycles represented in this history. But what else can we do? You need to start somewhere.

-ERD50
FWIW, I do think many posters here are overconfident about Firecalc. I'm not sure why you interpreted that as putting words in other's posts -- it's just my personal opinion and I expressed it as such. I do seem to read quite a few posts along the lines of obgyn65's famous "if Firecalc says you are good to go, you are good to go," which does seem (at least to me) overconfident.

Don't get me wrong -- I think FC is useful. But there are some methodology problems. It undersamples recent years. It doesn't account for changes in bond values due to interest rate fluctuations. The most important problem is, as mentioned, that it only models a few independent time series. If I run FC over a 30 year period, you might think that it is running 113 "experiments" (the number of 30 year periods between 1871 and 2013). But there is so much overlap between the time series (does 1960-1989 really give a different result than 1961-1990?) that the number of independent experiments is much less.

I agree that with traditional (normally distributed) MC, both temporal and inter-asset correlations can be lost. That's why I'm a fan of bootstrap statistics, since it allows you to keep those correlations. I'm too lazy to describe it in detail, but Wikipedia has a decent article on it. And the results (at least based on my own calculator) are more conservative than FC, which makes me feel better.
Fred123 is offline   Reply With Quote
Old 09-10-2014, 06:18 PM   #29
Give me a museum and I'll fill it. (Picasso) Give me a forum ...
REWahoo's Avatar
 
Join Date: Jun 2002
Location: Texas: No Country for Old Men
Posts: 50,004
Quote:
Originally Posted by Fred123 View Post
I do seem to read quite a few posts along the lines of obgyn65's famous "if Firecalc says you are good to go, you are good to go," which does seem (at least to me) overconfident.
__________________
Numbers is hard
REWahoo is offline   Reply With Quote
Old 09-10-2014, 06:22 PM   #30
Thinks s/he gets paid by the post
photoguy's Avatar
 
Join Date: Jun 2010
Posts: 2,301
Quote:
Originally Posted by brewer12345 View Post
But all those alternate realities are based on the Evil Dr. Pfau's favorite tool: made up numbers. So you don't like using the historical record and would rather base your plans on numbers Pfau essentially pulled out of his ass? Am I the only one who thinks this is problematic?
Actually I really do like using the historical record because it's less susceptible to researcher fudging and I would always look at this first. I was just saying that in terms of validation the MC researcher should put in assumptions that match the historical record and they should get results roughly the same as Firecalc. They might then explore other scenarios.

I guess my stance would be they both (historical and MC) can provide useful information but they also have problems. I'm very skeptical of some of Pfau's papers (e.g. the rising equity glide path one).


Quote:
Originally Posted by Fred123 View Post
For example, there aren't "a s***load of parameters" used, there are two (mean and variance) for normal or log-normal distributions and maybe one or two others if a "fat-tailed" distribution is used. There are no parameters used at all if bootstrap sampling is used.
Quote:
Originally Posted by ERD50 View Post
You say that MC uses only two parameters? But on how many variables? FC is looking at market returns, fixed returns, and inflation. I don't believe those are totally independent, so how does MC handle any correlations (positive or negative)? Without a full understanding of that, I don't know what to think about any MC run.
Take a look at this paper by Pfau "Asset Valuations and Safe Portfolio Withdrawal Rates"

Asset Valuations and Safe Portfolio Withdrawal Rates by David Blanchett, Michael S. Finke, Wade D. Pfau :: SSRN

To setup his MC simulation he has 5 different regression models for bond returns, stock returns, and inflation. I count 14 coefficients + mean/variance for 5 error terms (an additional 10 parameters) and this does not including variations on model structure or the hard upper and lower thresholds he puts in.


Quote:
But, more importantly, what are the alternatives? As a famous statistician once said: "all models are wrong, but some are useful."
I'm very fond of George Box's quote. I never meant to imply that MC methods were useless but they have their drawbacks. Same as Firecalc (except the drawbacks are different).


Quote:
Originally Posted by Fred123 View Post
I agree that with traditional (normally distributed) MC, both temporal and inter-asset correlations can be lost. That's why I'm a fan of bootstrap statistics, since it allows you to keep those correlations. I'm too lazy to describe it in detail, but Wikipedia has a decent article on it. And the results (at least based on my own calculator) are more conservative than FC, which makes me feel better.
I'm not sure how bootstrap would help here since you're still only resampling from a population of about 100pts. (i'm a huge of fan of resampling methods though).
photoguy is offline   Reply With Quote
Old 09-10-2014, 06:56 PM   #31
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
 
Join Date: Sep 2005
Location: Northern IL
Posts: 26,821
Quote:
Originally Posted by Fred123 View Post
FWIW, I do think many posters here are overconfident about Firecalc. I'm not sure why you interpreted that as putting words in other's posts -- it's just my personal opinion and I expressed it as such. ...

OK, you did say 'some', not 'most'. I might be a little sensitive and have a little pent-up stuff since someone recently ran with something I said and took it in a very demeaning direction, and that thread was closed before I had a chance to respond. Sorry.


Quote:
I do seem to read quite a few posts along the lines of obgyn65's famous "if Firecalc says you are good to go, you are good to go," which does seem (at least to me) overconfident.
REWahoo's emoticon is worth a thousand words. I'll just say that poster is so far from the mainstream here that anything they say is ... well, I better just stop there to keep myself out of trouble.


Quote:
Don't get me wrong -- I think FC is useful. But there are some methodology problems. ...
It's a tool. It provides some useful information. It can't do what isn't in its bag of tricks (hammers make bad saws). I mentioned the bond valuation issue, though I don't yet have a handle on how much that might throw off results (I need to research that). I suspect it isn't that big of a problem, only fairly small amounts get re-balanced, it probably washes over time to some degree, and the results with 100% EQ (which would not be affected) don't seem out of line with expectations. But it is an issue.

I'm not so sure the other issues are a big deal. Yes, there is only so much history, and much of it within the same cycle. But what can you do? Does randomizing those scenarios to create more of them really add much to the party? How do we know that the MC runs would look like anything that could have happened in history.

I'll need to read up on bootstrapping, but if I'm getting a hint of it from photoguy's post, I doubt I'll be impressed.

Bottom line, I just don't see how anyone can know that the MC runs represent anything useful. Some explanations seems circular to me - they compare them to history. Then what's the point, we have history?

But more importantly - is your MC giving you a more generous WR, or a more frugal one? If you are going to be more frugal, is it any different than using history and adding some cushion? If you are going to spend more than history says, doesn't that mean the MC didn't account for what we have already seen?

Maybe I'm missing something, but MC just strikes me as trying to tell me something that it just can't tell me (just like FC can't run a 50 year sequence starting 40 years ago).

-ERD50
ERD50 is offline   Reply With Quote
Old 09-10-2014, 07:35 PM   #32
Thinks s/he gets paid by the post
FIRE'd@51's Avatar
 
Join Date: Aug 2006
Posts: 2,433
If you look at Figure 1 in Pfau's paper you can see that, except for equity allocations below about 25%, the MC success rates and the Rolling History success rates are not all that different. Furthermore, I'm not sure I believe the 100% success rate for the Rolling History with equity allocations between 40% and 70%. I have never found a Firecalc result of 100% success using a 4% withdrawal rate, so perhaps the blue line should be lower, making the results even closer.
__________________
I'd rather be governed by the first one hundred names in the telephone book than the Harvard faculty - William F. Buckley
FIRE'd@51 is offline   Reply With Quote
Old 09-10-2014, 08:30 PM   #33
Recycles dryer sheets
 
Join Date: Jan 2014
Posts: 274
In spite of my missive below, note that I regard statistics as the Evil Math...

That said, MC analysis can be useful to gain insight into sensitivity of parameters. Most MC "draws" I've worked with have actually been based on historical performance of a parameter, using its descriptive statistics, e.g., mean, standard deviation. So, the historical performance of a stock, for example, can be used to develop a statistical draw function that returns values randomly selected from the bell curve of its historical data.

The historical record of the market is but one of what were a really large number of possible outcomes. But, other things could have happened, and the outcome would be very different. Well-constructed MC analysis helps you to effectively consider that possible variability in 'guessing' future performance.

With regard to FireCalc and I-ORP, I use both the historical and MC modes, and each tell me something useful. I spend more time regarding the historical mode reports, however, because I haven't picked at either's MC methodology to know what parameters are being MC-ed...
ggbutcher is offline   Reply With Quote
Old 09-10-2014, 09:01 PM   #34
Recycles dryer sheets
 
Join Date: May 2013
Posts: 127
Quote:
Originally Posted by photoguy View Post
I'm not sure how bootstrap would help here since you're still only resampling from a population of about 100pts. (i'm a huge of fan of resampling methods though).
Bootstrapping is sampling with replacement. So, in the example I gave (30 year cycles, from 1871-2013), if you're comparing to FC, you get a "sample space" of size 113**30 or about 10**61 vs 113 for FC. That's quite a few more experiments...
Fred123 is offline   Reply With Quote
Old 09-11-2014, 02:22 AM   #35
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
 
Join Date: Jul 2005
Posts: 6,115
historical data is to linked to only the results that happened before it. a great market runnup ala 1987 to 2003 could effect many many rolling time frame outcomes yet that run up of 14% for 17 years as an average may never happen again.

i like the idea monte carlo sims looked at the 10,000 possibilities of outcomes as well. the mere fact we have no historical time frame with low rates and high valuations at the same time makes any future historical sims unreliable at best if not dead wrong since no time frame existed to reflect where we are now.

in fact look at those poor results pfau got when accounting for the low yields in the monte carlo sims. even 30% stock looks poor in that run up. to be honest i believe that model may be the more useful tool right now for myself since i am retiring in 11 months 302 days 11 hours 12 minutes
mathjak107 is online now   Reply With Quote
Old 09-11-2014, 08:19 AM   #36
Thinks s/he gets paid by the post
FIRE'd@51's Avatar
 
Join Date: Aug 2006
Posts: 2,433
Quote:
Originally Posted by Fred123 View Post
Bootstrapping is sampling with replacement. So, in the example I gave (30 year cycles, from 1871-2013), if you're comparing to FC, you get a "sample space" of size 113**30 or about 10**61 vs 113 for FC. That's quite a few more experiments...
This approach makes a lot of sense to me. It seems to to capture the best of both approaches. There is no need to estimate a covariance matrix with all its associated problems, mainly the non-stationarity of the parameters. It retains the sequential correlations from market/inflation cycles built into Firecalc while reducing the serial correlation from data overlap. If I understand bootstrapping correctly, it makes all 30-year paths equally probable in a single drawing.

Can you tell us the difference in succes rates you get using Firecalc and the MC/bootstrap process for the same portfolio mix, e.g. a 75/25 equity/bond mix?
__________________
I'd rather be governed by the first one hundred names in the telephone book than the Harvard faculty - William F. Buckley
FIRE'd@51 is offline   Reply With Quote
Old 09-11-2014, 01:53 PM   #37
Recycles dryer sheets
 
Join Date: May 2013
Posts: 127
Quote:
Originally Posted by FIRE'd@51 View Post
Can you tell us the difference in succes rates you get using Firecalc and the MC/bootstrap process for the same portfolio mix, e.g. a 75/25 equity/bond mix?
I'll take a look in a few days -- heading off soon for a a beach backpack trip in Olympic Natl. Park. Retirement activities have to come first!
Fred123 is offline   Reply With Quote
Old 09-11-2014, 04:55 PM   #38
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
haha's Avatar
 
Join Date: Apr 2003
Location: Hooverville
Posts: 22,983
Quote:
Originally Posted by Fred123 View Post
I'll take a look in a few days -- heading off soon for a a beach backpack trip in Olympic Natl. Park. Retirement activities have to come first!
Give my regards to Shi-Shi.

Ha
__________________
"As a general rule, the more dangerous or inappropriate a conversation, the more interesting it is."-Scott Adams
haha is offline   Reply With Quote
Old 09-11-2014, 06:13 PM   #39
Thinks s/he gets paid by the post
photoguy's Avatar
 
Join Date: Jun 2010
Posts: 2,301
Quote:
Originally Posted by Fred123 View Post
Bootstrapping is sampling with replacement. So, in the example I gave (30 year cycles, from 1871-2013), if you're comparing to FC, you get a "sample space" of size 113**30 or about 10**61 vs 113 for FC. That's quite a few more experiments...
In bootstrapping you're just recombining the existing data to come up with alternate but similar histories.

One problem with bootstrapping is that if a phenomenon is not represented in the source data (or is inadequately represented) then all of the bootstrap samples will also be missing this behavior. So bootstrapping won't address Mathjak's concern of venturing into uncharted waters.
photoguy is offline   Reply With Quote
Old 09-11-2014, 06:26 PM   #40
Recycles dryer sheets
 
Join Date: May 2013
Posts: 127
Quote:
Originally Posted by haha View Post
Give my regards to Shi-Shi.

Ha
I will and good guess!
Fred123 is offline   Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Monte Carlo question modlair FIRECalc support 2 05-26-2006 06:51 PM
Monte Carlo Excel Add-In haha FIRE and Money 4 05-23-2006 10:51 AM
monte carlo Mysto FIRE and Money 5 04-18-2006 06:34 PM
Monte Carlo now can be used by brokers newellcr FIRE and Money 3 02-18-2005 08:41 AM
Online Monte Carlo calculator RoyUtah FIRE and Money 5 01-18-2004 01:41 AM

» Quick Links

 
All times are GMT -6. The time now is 02:32 AM.
 
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2024, vBulletin Solutions, Inc.