Suggestion: Sensitivity Analysis

[modlair]

Accept or reject as you wish, sir:

Many SWR discussions I've read talk about the results of 4-5% being dependent on history and then debate begins about whether or not the future will be more bullish or more bearish than the past.

Perhaps a sensitivity analysis in Firecalc has merit? Meaning, someone who wants to see the effect of a more bearish future (than the past) could bias all historical numbers to be more bearish on average by X%. Or more bullish by X%.

One can then start to draw conclusions about how sensitive an SWR's success % is to a future that is biased X% up or down wrt history.

Shrug.

[Nords]

Quote:

Originally Posted by rodmail Perhaps a sensitivity analysis in Firecalc has merit?* Meaning, someone who wants to see the effect of a more bearish future (than the past) could bias all historical numbers to be more bearish on average by X%.* Or more bullish by X%.

Hacker's sensitivity analysis:
Start with the SWR that produces a 95% success rate and raise it until the success rate declines to 50%.

[Rich_by_the_Bay]

Quote:

Originally Posted by rodmail Accept or reject as you wish, sir: Many SWR discussions I've read talk about the results of 4-5% being dependent on history and then debate begins about whether or not the future will be more bullish or more bearish than the past. Perhaps a sensitivity analysis in Firecalc has merit? Meaning, someone who wants to see the effect of a more bearish future (than the past) could bias all historical numbers to be more bearish on average by X%. Or more bullish by X%.

This is great idea. The same concept can be emulated as Nords says by framing your results until you reach the probability you are comfortable with but it would be tedious.

A REALLY useful feature would be to generate a 95% confidence interval around the results, e.g. probability of succes = 92% (CI95 = 85% to 97%). That is how I get the best sense of how usable the results are for me.

Really, though, it is unrealistic to expect poor Dory to do this, and most users who are not hopeless statistics-geeks like me would not value it much.

I really love the upgrades so far. Thanks, Dory.

[dory36]

Also, if you select one of the options that iterates for a given success rate, or the one that lets you see the effects of deferring retirement, the chart will show you a rudimentary sensitivity chart.

[modlair]

Quote:

This is great idea. The same concept can be emulated as Nords says by framing your results until you reach the probability you are comfortable with but it would be tedious.

Well, maybe not so great.

The alternate suggestion explores the effect on success of varying SWR, which is sort of just automating what people do manually now.

I was interested in how sensitive a given SWR's success is to the future varying in performance wrt the past.

But now that I think about this, maybe we already have it? If I think the future will be 2% worse than the past, is it legit to simply set the AER to 2.18? And if more bullish then -1.82? I checked, you can enter a negative AER.

Maybe this can be handled with just some text next to the fund expense 0.18 number.

[modlair]

Okay, wait a minute. I just crunched a few runs to explore the effect of bias on returns using the Annual Expense Ratio value that defaults to 0.18%. I nudged it to 1.18 and 2.18 etc. The effect is enormous. Adding 1% to that (simulating 1% worse returns than history) reduces a given SWR success number from 95% to 82%. Another 1% bias on returns takes it down to 70%. This is a very sensitive parameter.

But.

I think this is bogus. That number is also presumably changing the returns on the fixed income portion of the 55/45 mix. I think what we want is something that just defines S&P bias. In fact, one could make a strong case for every 1% bias on the S&P should adjust the rate bias on fixed income inversely.

[dory36]

Yes, the expense ratio is for the whole portfolio. And yes, it is a very sensitive number, as when you move 1-2% up or down, and the returns are often in the 5% range, then you're swinging performance by a huge amount.

But I'm not sure I follow re moving fixed income expenses inversely with equity expenses.

Or are you talking solely about exploring a different question, using this as a way to do so?

[modlair]

No Dory, it's still the original issue. The ability to explore an opinion that the future will be more or less bullish than the past would need to be a number applied only to the S&P.

I tried to do that with current capability by using the AER, but that's bogus for the reasons you just confirmed.

I'm only musing about inversely adjusting fixed income vs S&P because of a popular presumption that the market climbs when rates drop, and vice versa. When there is worry that the Fed is not done raising (in the current cycle), the market drops.

But I think that if you are willing to add this feature, the way to do it would be seperately for equity and bonds. If someone thinks the market will do better than historically (maybe because of the productivity available from computers in general) he can add 1% to just the equity portion of the mix. And vice versa. If you like, provide the same option for the fixed income (bonds) portion as well. Maybe someone thinks rates will be higher than in the past, or lower.

I suspect it's easy coding to do and sensitivity analysis is a good thing for this parameter.

[Nords]

Quote:

Originally Posted by rodmail I suspect it's easy coding to do

Well, if it was hard then he'd do it, but since it's easy then you can pick up your coding assignment from Dory tomorrow!

[dory36]

This may need to be one of those things on the list for more careful study rather than just jumping in and doing it.

Probably showing I am not quite following the suggestion, here are some questions that immediately jump to mind when I think of making the change:

If the S&P growth each year is increased by a user-defined amount, is the dividend yield reduced to maintain the same doillar dividends, or are the dividends increased also?

Is the change additive or a factor? If additive, it obviously must be computed each year based on the magnitude of the underlying value. If a factor, wouldn't the increase in volatility create a whole new set of issues?

Several other questions are percolating, and I'd have to do more work than is allowed on Saturday afternoons to figure out if they are real or imagined, and if real, to articulate them.

Maybe if you walk through a step by step flow of the calculations..

dory36

[Nords]

Quote:

Originally Posted by dory36 If the S&P growth each year is increased by a user-defined amount, is the dividend yield reduced to maintain the same doillar dividends, or are the dividends increased also? Is the change additive or a factor? If additive, it obviously must be computed each year based on the magnitude of the underlying value. If a factor, wouldn't the increase in volatility create a whole new set of issues?

Bogle's blog has a speech from last month that breaks stock returns down into dividend return, growth return, and speculative return.

http://www.johncbogle.com/speeches/J...int%204-06.pdf

Maybe dividend & growth should be held at their historical values and the speculative return could be added (not multiplied).

[dory36]

I like your idea.

Please bring the commented code to tomorrow's meeting. Thanks!

[modlair]

Quote:

Maybe if you walk through a step by step flow of the calculations..

Fair enough, and I certainly don't want to distract from other work. But your core question is EXCELLENT. It focuses on the fact that a 2% up bias in returns does NOT mean we simply increase the yearly index by 2%.

The theory is simply that in discussion of SWR, people say that 4% yielding safety really has to be 3% because returns of the future will be lower. Or they say the reverse. The question is, what impact on SWR safety does a bias upward or downward on historical equity returns have? The focal word in that paragraph is "returns".

This will NOT be simply saying . . . I think returns in the future will be 2% better than the past and therefore all that need be done is take the index value from 1871 thru 2005 and increment each year by 2%. That is wrong. That is not an increase in returns. That is an increase in the index. It is the wrong derivative in calculus.

I therefore think that 2% will have to compound over each 135 yearly value. It is returns that tell the tale for us, not the value of the index used to measure equity performance.

I was under the impression that the equity performance for Firecalc each year since 1871 (or whatever) is a published number from a historical database. That database is presumably total return (including dividends) -- meaning when one says the S&P did +5% in 2005, that includes dividends. The Dec 31 S&P index number should be 5% greater than Jan 1. The keepers of the index include dividends, I think so you would have nothing to address because their effect is in the published number.

If what you have is a closing price for an index since 1871, and not a database of yearly returns, then the return must be derived. After that we'd need:


Something like:

Return[1872] = (Index[1872] - Index[1871]) / Index[1871]
/* this computes 1872's stock market return */

BiasedReturn[1872] = Return[1872] + UserBias
/* This biases that return upward or downward */

BiasedIndex[1872] = BiasedIndex[1871] * (1.0 + BiasedReturn[1872])
/* This creates a new index. 1872's value would have been something different if the bias had been present. Note it is based on the BiasedIndex value of the previous year, because this effect will compound */


Somewhere above this loop you'd have a BiasedIndex[1871] = Index[1871] because it is the first datapoint and needs initializing. Replace the year numbers above by an i counter in a loop i = 1871, i =< 2005, i++ and you're good to go.

One thus computes the raw return of 1872 and other years from the raw data. One adds user bias to the raw return and computes a "new" index based on returns computed from the original, but with those returns biased. The biased index (or returns, however you've mechanized the code) is what is applied to the user's money.

I suggest compounding here because if you just increased each index year by 2%, the 1872 number would be 2% higher and so would the 1871 number. The rate of increase from 1871 to 1872 would not be 2% higher and we want it to be.

Does this make sense?

[cute fuzzy bunny]

Quote:

Originally Posted by dory36 Please bring the commented code to tomorrow's meeting.

Can I skip tomorrows meeting? I hate meetings.

I've got some holes to dig in the yard anyhow.

[wabmester]

Wouldn't this defeat the point of FIREcalc? It's purely a historical sanity check. The historical results aren't just the function of annualized returns being high or low. The results are a function of the very specific historical sequences. The returns, the volatility, the duration of bull and bear cycles, the regression to the mean, etc.

Don't mess with it. The sort of tuning you're talking about is ideally done with MC simulators.

[modlair]

Quote:

Don't mess with it. The sort of tuning you're talking about is ideally done with MC simulators.

If MC simulators were meaningful, biased or not biased, would the Trinity study ever have been done, and firecalc thereafter?

[wabmester]

Quote:

Originally Posted by rodmail If MC simulators were meaningful, biased or not biased, would the Trinity study ever have been done, and firecalc thereafter?

Of course MC simulators are meaningful.* *The Trinity Study and FIREcalc simply take another approach to the question of portfolios surviving various conditions.

The question FIREcalc answers is "how would this portfolio have done historically?"* *That's all it can tell you. [1]

MC simulators can also tell you how your portfolio would do through a bunch of alternative histories, generally much longer than 120 years each.* * One approach isn't better or worse.* *Just different, but FIREcalc is "pure" in that it uses historical sequences exclusively, so you would be diluting that if you tried to parameterize the results.

Taking historical "patterns" and fiddling with various parameters is exactly what MC simulators were designed to do.

Edit:
[1] Of course, it does much more than this.* *It'll tell you how your portfolio would have done combined with a simple algorithmic withdrawal strategy.* *But you get the idea -- don't mess with history!

[modlair]

Well, I guess the only rebuttal is that lots of tweaking is already in firecalc.

History can be abbreviated by choosing a year other than 1871 to start.

History didn't include mutual funds at 0.18% cost of investment. History had only full service brokers. So a test being performed with 0.18 expense is an approach not historically accurate. If it had been present in those years, might that have changed market behavior? Probably. So it is a tweak.

History is changed from the new firecalc vs the old one in that the final several years are truncated off the solution. Those years occurred, but they are not used.

I guess I don't understand the objection. The suggested mod is an option. I would certainly not want a default to be some skew of the historical data. If it's easy to code, why deny it to folks?

[wabmester]

Quote:

Originally Posted by rodmail I guess I don't understand the objection.* The suggested mod is an option.* I would certainly not want a default to be some skew of the historical data.* If it's easy to code, why deny it to folks?

I guess I never liked the swiss-army knife approach to anything.* *Give me a wrench or give me a hammer, and keep your wrammer.*

Seriously, I think the mod would be meaningless.* *Why pick on the return amplitude?* *Why not stretch or contract the length of bull/bear cycles?* *Why not move sequences around to see what happens when you don't get as much RTM?

I think a calc that let you do all sorts of these manipulations would be way cool, but I wouldn't recognize it as having any resemblence to FIREcalc or the study upon which it is based.

[modlair]

Quote:

Seriously, I think the mod would be meaningless. Why pick on the return amplitude?

Because that is the matter under discussion. "Yeah, sure, you are getting 4.5% SWR with historical data, but suppose returns in the future aren't as good -- because the 20th Century was the American century. What happens if the next 30-40 yrs is weaker?"

Or . . . "Yeah, sure, you are getting 4.5% SWR but I think I could take more than that out and live a more lavish lifestyle because that 4.5% is based on a century where the investing public did not have access to the body of knowledge now present. That historical data is based on business that did not have the productivity of computers. I think the returns will be better in years to come. What happens to my SWR if they are better by X%?"

Quote:

Why not stretch or contract the length of bull/bear cycles?

Because there is no algorithm, and certainly not one easy to code. You would need a Fast Fourier Transform applied to the data to extract the frequencies present in the waveform, filter the result in a low or high pass digital recursion and then imagine that you really did stretch or contract the wavelength of cycles that may or may not have even been present.

A simple bias of the returns up or down is a pretty reasonable answer to provide a person's idle question of what happens if returns are better or worse.

Quote:

Why not move sequences around to see what happens when you don't get as much RTM?

Because it's harder to code, one would think, though that would be interesting.

I just offered a suggestion that I encounter often in discussion and one that I thought would be easy to implement. Shrug.