Suggestion: Sensitivity Analysis

[wabmester]

Originally Posted by rodmail

"Yeah, sure, you are getting 4.5% SWR with historical data, but suppose returns in the future aren't as good -- because the 20th Century was the American century.* What happens if the next 30-40 yrs is weaker?"

Well, let me say that I'm not trying to impose my view on anybody.* * If the feature were added, I might even play with it.* *It's just that I'd have no clue what the result meant.

The problem is that we don't know the future, so any mod that tries to make things better or worse in some single dimension is really meaningless.* * It gives us no new information.* *We still don't know the future.* *What do you think we'll learn by playing with the historical returns?

And, as I hinted at above, it's a slippery slope.* *Is it weaker returns that would really hurt a potential retiree?* * Ask any retiree, and they'll tell you that volatility and inflation are the real killers.* *So, let's add some more features to play with those parameters.* * The result is still meaningless.

My core point is that coming up with some *arbitrary* case that is worse than any existing worst-case is meaningless.* * If you lower the historical returns, or increase the historical volatility, or change the historical RTM, these will all lead to a predictable result: a lower SWR.* * But we can be pretty certain that our pretend scenario will have no resemblence whatsoever to our future, so it does us no good at all.

You raise a good point, though.* * I would love a tool that let me model various future scenarios.* *I'm not sure what I'd use the results for, but it'd be a fun video game.*

[modlair]

Oh, I'm totally okay with your points. They are valid questions.

The problem is that we don't know the future, so any mod that tries to make things better or worse in some single dimension is really meaningless. It gives us no new information. We still don't know the future. What do you think we'll learn by playing with the historical returns?

But this is why Sensitivity Analyses are done for all sorts of planning processes. When you don't know what the future holds, then it would be very useful to discover how sensitive some parameter is to another. The classic case is elasticity of demand, where one explores the impact of price on units sold. You move numbers around in test markets, gather data, plot the result. Intuition sometimes fools you. There have been cases where an increase in price increased units sold -- because a certain cachet was lent to the product.

But no matter. The point of Sensitivity Analysis in this context . . . is that maybe it takes a full 8% of market bias upward to move the % of successful runs at 4.5% from 95% to 96%. Would not such a discovery of profound insensitivity of success % to returns be useful to someone's planning? I would certainly get very conservative in my equity choices if I had data like that.

In statistics it is called ANOVA for analysis of variance. Dependent variables, by definition, vary as a function of independent variables. But the degree to which they do is not known until calculated.

Up to Dory, of course.

[wabmester]

Have you already browsed some of the information at intercst's site?

http://www.retireearlyhomepage.com/safesum.html

He has varied several parameters to evaluate sensitivity.

Fun to play with, but here's what keeps sticking with me:* if you simply put your money under your mattress, the SWR for 30 years is 3.33% with no inflation adjustment (100%/30).* *Instead of putting the money under your mattress, invest in something that merely keeps up with inflation and you'll do fine at that SWR.* * Put the money in something like TIPS, and you get both the real returns + your nest egg depletion.

So, if you take volatility out of the equation, it's obvious that the SWR just scales linearly with returns.

If you want an interesting sensitivity analysis, I think some of the other things we discussed (volatility, cycle length, RTM amplitude, etc) are where the results start to get surprising.

[modlair]

If you want an interesting sensitivity analysis, I think some of the other things we discussed (volatility, cycle length, RTM amplitude, etc) are where the results start to get surprising.

Well, I'm a somewhat low level engineering executive trying to wind down to RE, so I have a bit of time now and then and would be willing to help -- perhaps algorithmically.

The items you've just mentioned -- there will be a challenge defining them for a codable algorithm.

For example, if you wanted to test the sensitivity of SWR success to future market volatility change, how do you do that. I suppose you take the data set of 135 values of the market index, compute a sloped mean, then the standard deviation around that upward sloping mean -- and then somehow, in entirely arbitrary fashion, change the values of the 135 points to yield a different value of 1 sigma. Yes, you can change the points to yield a 20% increase in sigma, but which points do you change to do that?

For cycle length, as I mentioned, you have to compute a Fast Fourier Transform to identify the sinusoidal frequencies in the data stream. Once you have them, you say, okay if I have lower frequencies the cycle wavelengths are longer, and vice versa. So to make the cycles be longer, you filter out the high frequencies, and then take that frequency domain data and inverse FFT it back to time domain.

Guy, that is not trivial coding. And when you're done you have to recognize that you forced a definition of sinusoidal behavior onto a datastream that may not be sinusoidal.

As for how powerfully a datastream tends to regress to a mean, well, that's a multi pole digital low pass filter. Coding multiple poles will be one hell of a lot of functions, with each additional pole commanded by a user declaring he wants regression to be more or less powerful. What the low pass filter will do is compress volatility to zero around either the mean of the overall data, or the mean of a slope of multi year market trend. Again, this is an almost undefinable algorithm.

I don't object to trying these things and I'd even be willing to help algorithmically. My original suggestion was not intended to focus on the superiority of future market performance bias wrt the past as the best candidate for SA. It was a combination. The combo was that having that feature available would be valuable to people who ask that idle question in discussions about SWR, but also it should be really easy to 1) define and 2) code.

[wabmester]

That actually sounds like a cool approach.* *I'd be surprised if somebody hadn't already run an FFT on the market gyrations.* *Maybe I'll poke around to see what I can find.

If it were me, I'd try to fit the data to a Bezier spline, then you could rubberband the thing all over the place, including stretching about Y to shorten/lengthen cycles and stretching about X to vary return amplitude.* * I have to admit, it's been 20+ years since I coded anything like that, but it sounds like fun.

[BigMoneyJim]

I've toyed with the idea of making a calculator. I'm a bit more serious about a rent vs. buy calculator since that is a more immediate question for me, and that would more or less require contrived future variances, and a rubber bezier curve is once concept I had in mind.

I'm moving the weekend of June 10, and after that I will be job hunting, visiting family and fixing up my mother's house. If I have the time and motivation I may start seeing what I can start...that is if dory36 doesn't beat me to it. He's on a tear lately!

I posted this partially to indicate my interest and partially to make this thread easier for me to find later. We'll see what happens in June.

[astromeria]

Sorry, couldn't resist....

INCHWORM

Inchworm, inchworm
Measuring the marigold
You and your arithmatic
You'll probably go far

Inchworm, inchworm
Measuring the marigold
Seems to me you'd stop and see
How beautiful they are

Two and two are four
Four and four are eight
Eight and eight are sixteen
Sixteen and sixteen are thirty-two

I can imagine Danny Kaye singing it, but I can't remember the context (a musical?). It was a popular song in the early 50s IIRC, at least around my house.

[laurencewill]

Ow, my head hurts! But I was left with an impression Wab touched on, why not just change the makeup of the portfolio wrt equities/bonds to see deltas in SWR and it's dependency on return? A bit tedious to get a rough sketch of a sensitivity analysis, but...

[wabmester]

I played with the new FIREcalc a bit.* * Way too many parameters already.* *I want freedom from choice.* *Give me back the old version that just spit out "4%" no matter what I put in.* *

[modlair]

Ow, my head hurts! But I was left with an impression Wab touched on, why not just change the makeup of the portfolio wrt equities/bonds to see deltas in SWR and it's dependency on return? A bit tedious to get a rough sketch of a sensitivity analysis, but...

Ooh, nice try. That had potential for a few minutes. But nope. The problem is the bond portion of the mix changes its rate as the equity portion changes its rate. And worst of all, you can make a case for them not being independent. As bond rates rise, maybe the stock market drops.

So if you try to emulate a future bias of stock market performance wrt the past by nudging your asset mix, you don't get the desired experiment. Not desired result; desired experiment. In other words, raising bond mix to emulate a worse future stock market return wrt the past is not valid.

I played with the new FIREcalc a bit. Way too many parameters already. I want freedom from choice. Give me back the old version that just spit out "4%" no matter what I put in.

Ha. Given the other thread this is really funny. We're tracking down why old firecalc results don't equal new firecalc results using the same inputs.

And I have bad news. Pretty much all results show the new firecalc decreasing your SWR and those sources of variance concretely explained so far point at the old firecalc being improperly optimistic.

Sigh.

[dory36]

Originally Posted by rodmail

And I have bad news.* Pretty much all results show the new firecalc decreasing your SWR and those sources of variance concretely explained so far point at the old firecalc being improperly optimistic.

Especially if you ignored the narrative with the old one...

[wabmester]

Originally Posted by rodmail

Ha.* Given the other thread this is really funny.* We're tracking down why old firecalc results don't equal new firecalc results using the same inputs.*

And I have bad news.* Pretty much all results show the new firecalc decreasing your SWR and those sources of variance concretely explained so far point at the old firecalc being improperly optimistic.

I was joking, but consider what we found in the other thread. * I was just doing a baseline sanity check, using exactly the same parameters, but used two different sources for long-term treasury bond data in the new FIREcalc. * *Got two different SWRs. * One was 15% higher than the other. * *And that was after eliminating every other possible unknown.

So, in the end, I prefer just sticking with a ballpark of 4% because the results you get from FIREcalc are only meaningful if the future plays out *exactly* the same way it did in the past. * And the chances of that happening are zero.

(And, sorry, it was my fault that there were two narratives in the other thread.* *The two data sources dory36 was talking about are both part of the new FIREcalc.)

[dory36]

Originally Posted by wab

I played with the new FIREcalc a bit.* * Way too many parameters already.* *I want freedom from choice.* *Give me back the old version that just spit out "4%" no matter what I put in.* *

I have reserved this version for only special users.

Please visit http://firecalc.com/simple.php

But keep it to yourself!

[wabmester]

I like it! Reminds me of my old Unix days. Simple is beautiful.

[modlair]

I don't like it. It is improperly simple because of the definition of "normal". "Simple" should be defined as . . . the default for all parameters is what most people will find is correct for them.

Most people will collect some amount of Social Security. If "most" was 51% then okay, I'd be more comfortable with leaving it out. But it's not going to be 51%. It's more likely 90%. I suspect there is no SS inclusion in that simple approach so it is just not "simple" and SS has a big effect on SWR.

The way-too-often-quoted 4% SWR is wrong for most people. SS will move SWR upward.

[laurencewill]

rod, you forgot the smiley face.

[modlair]

Ha!

[Nia4914]

Originally Posted by rodmail

Well, maybe not so great. 

The alternate suggestion explores the effect on success of varying SWR, which is sort of just automating what people do manually now.

I was interested in how sensitive a given SWR's success is to the future varying in performance wrt the past.

But now that I think about this, maybe we already have it?  If I think the future will be 2% worse than the past, is it legit to simply set the AER to 2.18?  And if more bullish then -1.82?  I checked, you can enter a negative AER.

Maybe this can be handled with just some text next to the fund expense 0.18 number.

"about" is wrong!