Market behavior is near-random*. That's settled science.

OldShooter said:
The title is careful to say "near-random." Proof that the market is not perfectly random is as simple as observing that the distribution is biased, which I did in the post. The existence of momentum, also pretty well settled science, also proves that it is not perfectly random. The Andrew Lo book, which I have, also serves.

The point is that not-quite-perfect randomness is proven well enough to explain most of what we as investors see and the decisions we have to make. Inability to time the market is probably the most important. It's the other side of the predicting-the-market coin.

I have deliberately avoided the Efficient Market Hypotheses because it is not necessary to support the assertion and it is hardly a settled matter. (For those interested, here is a 40 minute video that I enjoy and learn from every time I look at it. Two Nobel Prize winners discussing the topic. https://review.chicagobooth.edu/economics/2016/video/are-markets-efficient)

That is not to say that it is impossible for little investors to nibble around the edges of imperfect randomness and maybe get a chunk of cookie from time to time. @NW-Bound has mentioned a few. Occasionally winning a momentum bet, taking advantage of the market's imperfect reaction to earning announcements, and occasionally succeeding in exploiting an overreaction to an event. This is sort of protected territory for the little guy IMO because his tiny money does not move the market. A professional running a significant portfolio must make large enough trades that they will reduce or eliminate his success even if he sees the same little things that the little guy sees. Which he probably does.

But the tricks that a small investor might successfully execute don't, I think, matter for the vast majority of investors on this site because they are not equipped or inclined to do them. For that majority, accepting real world randomness should help them understand why betting on sectors increases risk, why chasing successful managers is doomed, and why the best policy is to completely ignore the talking heads, click-seekers, and hawkers of expensive mutual funds.
Click to expand...

+1

This is one of the best descriptions on how the market works for small investors that I have read.
 
Ed_The_Gypsy said:
Another way to look at it is, there is a lot of noise in the signal, but a low-pass filter shows an increase over time.
Click to expand...
Nice. Yes. I guess I never thought of myself as a low pass filter though. :LOL:
 
I try to be a Kalman filter, in an attempt to filter out signals amidst the noise.

PS. Kalman filter is a misnomer, which unfortunately sticks. It should be called a Kalman state estimator.

PPS. A Kalman state estimator is proven to be optimal, meaning producing the best results possible, but only for Gaussian noise. When the noise is not Gaussian, or the model is not accurately determined, it can lead to disastrous results. Skilled Kalman filter designers routinely employ specialized ad-hoc processing to protect against anomalous errors. This extra added step is proprietary, and can make or break a system. It can make the results worse!
 
Last edited:
OldShooter said:
There are about 10,000 mutual funds in the US; if we set 10,000 monkeys to flipping coins, after the 8th toss about 40 of them will have flipped ten heads in a row.
Click to expand...


I don't understand this part. How could 40 monkeys flip ten heads in a row after only 8 tosses?
 
The above example about monkeys tossing coins is very popular, and told often. However, it is not a good argument, although it is statistically correct and sounds compelling.

Suppose an investor trails the market 2 out of 3 years. You then say his record is dismal for winning only 1 out of 3 times. However, suppose that in the 2 years that he loses he is only 1% behind in each year, but in the single year that he wins he is 10% ahead. Overall, he stills wins. The frequency of winning/losing is not the whole story. It is how much you win and how much you lose each time.

Nobody can beat the market every year. And you do not want to do that either. When the market is frothy and loaded with dotcoms as in the late 90s, the only way you can beat a crazy market is to be even crazier. You would have to go on margin on the dotcoms.
 
Last edited:
Fermion said:
I don't understand this part. How could 40 monkeys flip ten heads in a row after only 8 tosses?
Click to expand...
It'sjust the math. Start with 10,000 monkeys standing (there are about 10,000 mutual funds), have them flip their coin and sit down if it comes up tails. This leaves about 5000 monkeys standing. Flip again, 2500 standing and so on. You can make a little 8-line spreadsheet and see it all.

& @NW-Bound is right, it's not a perfect metaphor. And its also true that about 40 of the sitting monkeys got seven in a row. At the other end, about 7% of funds are merged or closed every year. I haven't done the math but I'd guess these are equivalent to the unlucky monkeys who got zero, one, or two heads in eight trials.

There's a common story that makes a similar point. Try here: https://civilstat.com/2011/11/flipping-out/ and read beginning at the "Coin Trick" paragraph.

But the point is that stock pickers with above-average results do not disprove the near-randomness assertion. Another way fund marketers mask the randomness is with incubated funds: https://www.investopedia.com/terms/i/incubatedfund.asp and by ignoring survivorship bias: https://www.investopedia.com/terms/s/survivorshipbias.asp

@NW-Bound is also correct in his hypothetical where a big win wipes out a few small losses. I don't have the stats at hand, but I have read that real-world stockpickers that outperform typically do it by only a few percent while the underperformers lose more. This has to be the case at the macro level where decades of evidence show that most stock-pickers underperform.

The key to handling all this, of course, would be to identify the winners ahead of time instead of in the rear-view mirror. Unfortunately no one has figured out how to do this (or is keeping very, very quiet.) Here's a 6-minute video on that subject: https://famafrench.dimensional.com/videos/identifying-superior-managers.aspx
 
Still, not a one of those 10K monkeys got 10 heads in 8 flips. :)
 
donheff said:
Still, not a one of those 10K monkeys got 10 heads in 8 flips. :)
Click to expand...


As I mentioned, some of the monkeys cheated and flipped more times than they were allowed. :)


OldShooter said:
But the point is that stock pickers with above-average results do not disprove the near-randomness assertion. Another way fund marketers mask the randomness is with incubated funds: https://www.investopedia.com/terms/i/incubatedfund.asp and by ignoring survivorship bias: https://www.investopedia.com/terms/s/survivorshipbias.asp

I don't have the stats at hand, but I have read that real-world stockpickers that outperform typically do it by only a few percent while the underperformers lose more. This has to be the case at the macro level where decades of evidence show that most stock-pickers underperform...
Click to expand...



I don't disagree with the above. However, I do not think it is because MF managers are all dumb. The problem is that their shareholders demand that they beat the market all the time, year in/year out. When the market surges due to frothy conditions, if they shun the "hot" stocks who they fear will implode soon, they will trail the market and their shareholders bail out of their funds in drove. Right now, they all hoard up the FAANG stocks.

I have told this story, but it is worth repeating. During the 1999-2000 stock mania, I had been an active investor for only a few years (I was all in MFs before that), and felt something very wrong. All the stocks I bought went up, some a lot more than others but they all went up. Surely I could not be such a genius. I looked into the MFs that I held, and they all had these same stocks. Hah, it was that easy. That was when I researched and read "A Random Walk down Wall Street" by Burton Malkiel.

Also at that time, I read many finance magazines. In one of them, Business Week I believe, I read a story told by a MF manager. He said that he was fearful of market collapsing, and sold some shares to raise cash level. His shareholders were upset. He said one of them called in to tell him that if the shareholder wanted to keep cash, the money would not be sent to the MF in the first place. "Buy, buy, buy" was the mantra.

What I took away from that is this: if I wanted to avoid bubbles, I had to do it myself.

And with time, I have found that some non-index funds also perform well and manage to survive financial crises well. They tend to be balanced funds. But if you expect them to beat the market every single year, no they never promise to do that.


PS. During the stock runup in early 2000, the market scared me and I was losing sleep even though I "made" money nearly every day. I later learned that the simple way to fix that insomnia was to sell until I felt safe again. For people who forget, from Aug 2000 to Sep 2002, $1 in the S&P became 55c.

In the 2008-2009 Great Recession, I slept very well, having been through a similarly scary time only a few years earlier.
 
Last edited:
I think that there is some "settled science". Don't we all think that in 5, 10, or 20 years, that the markets will be higher than today? We all "know" that things be better in 5,10, or 20 than now. ( I'm leaving out those who say the planet is doomed in 11.5 years.) And while the market will have significant ups and downs in between, we all expect it to be higher.
 
The market may exhibit randomness in the short term (greed, fear, emotions, politics, etc.), but markets are highly correlated to earnings in the long run, that’s why investors are in for the long term. And using the track record of active managers as evidence the market is random is apple and oranges. That’s why passive index investing has become so popular.
 

Attachments

  • 846F41C7-F473-4372-817C-1F709FFFD9D6.jpeg
    846F41C7-F473-4372-817C-1F709FFFD9D6.jpeg
    324.2 KB · Views: 22
OldShooter said:
The title is careful to say "near-random." Proof that the market is not perfectly random is as simple as observing that the distribution is biased, which I did in the post. The existence of momentum, also pretty well settled science, also proves that it is not perfectly random. The Andrew Lo book, which I have, also serves.

The point is that not-quite-perfect randomness is proven well enough to explain most of what we as investors see and the decisions we have to make. Inability to time the market is probably the most important. It's the other side of the predicting-the-market coin.

I have deliberately avoided the Efficient Market Hypotheses because it is not necessary to support the assertion and it is hardly a settled matter. (For those interested, here is a 40 minute video that I enjoy and learn from every time I look at it. Two Nobel Prize winners discussing the topic. https://review.chicagobooth.edu/economics/2016/video/are-markets-efficient)

That is not to say that it is impossible for little investors to nibble around the edges of imperfect randomness and maybe get a chunk of cookie from time to time. @NW-Bound has mentioned a few. Occasionally winning a momentum bet, taking advantage of the market's imperfect reaction to earning announcements, and occasionally succeeding in exploiting an overreaction to an event. This is sort of protected territory for the little guy IMO because his tiny money does not move the market. A professional running a significant portfolio must make large enough trades that they will reduce or eliminate his success even if he sees the same little things that the little guy sees. Which he probably does.

But the tricks that a small investor might successfully execute don't, I think, matter for the vast majority of investors on this site because they are not equipped or inclined to do them. For that majority, accepting real world randomness should help them understand why betting on sectors increases risk, why chasing successful managers is doomed, and why the best policy is to completely ignore the talking heads, click-seekers, and hawkers of expensive mutual funds.
Click to expand...

Old Shooter-Is your premise even a scientific statement? "Near random" is not defined. Any evidence presented to the contrary can be explained away by the statement "I said near-random".

Above you identify evidence that financial markets are NOT random, which you accept. So that leaves us quibbling over the definition of "near" I guess.

A more defendable statement might be "Financial markets exhibit some characteristics of randomness". But without the words "settled science" in there, it might not meet your objectives ;).
 
Market behavior is not random, but it is driven by the professional money managers, traders and investors and it is foolish for an individual casual investor to believe that they can develop any insight for how financial markets will perform over any particular period of time. The best such investors can hope for is to ride the wave and hope the law of averages works out ok in the long run.
 
While I would agree there is randomness to its movements along the way, if it was truely random it would not increase over the long term. And since most every market does increase over the long term (even with 20-30 year kilos in some countries); it is ultimately driven by economic fundamentals.

What this means to me is that stock picking and timing the market are fools errands pitched to the human desire for “control”.

Edit: Re read the original post in thread. It basically said some of what I said here, but had what I consider a misleading headline. Kind of like news publishers do sometimes :)
 
Last edited:
coveredbridge said:
Market behavior is not random, but it is driven by the professional money managers, traders and investors and it is foolish for an individual casual investor to believe that they can develop any insight for how financial markets will perform over any particular period of time. The best such investors can hope for is to ride the wave and hope the law of averages works out ok in the long run.
Click to expand...
I have no disagreement with that. The randomness is introduced at the next level up where the pros are getting stirred up by the random and unpredictable things are constantly occurring: a general gets assassinated, earnings reports include surprises, oil producers increase or decrease production, countries' GDP numbers come out, Korea and Japan get into a pi$$ing contest, .... The list goes on and on.

Midpack said:
Care to comment on the randomness of the chart before we go off on tangents?
Click to expand...
Sure. Your chart pretty much shows graphically what I said in the OP: I say "near-random" because [if] the market were completely random (classical Gaussian distribution) the deviations would be centered on zero and there would be no point in investing because the markets would never change. Actually the distribution is centered a few percent to the right/positive, resulting in what we have seen in a hundred years of market history: a slow, steady trend interrupted frequently by random excursions. That's why buying and holding a diversified portfolio works. The buy and hold investor basically believes in and rides the trend, ignoring the noisy excursions.

Your chart has a linear Y-axis, so it also includes the effect of compounding -- the hockey stick to the right. Often this type of chart has a logarithmic Y-axis which better isolates the steady trend. For example:

38349-albums210-picture1776.jpg

​
 
When people talk about the randomness of the market, of course they mean the day-to-day variation that swamps out the long-term trend that is really bitty compared to the former.

For example, today the S&P opened down more -0.44% below yesterday close, fluctuated, then ended up closing +0.35% up.

The above range of 0.79% is nothing compared to what we have seen in some really bad days in the past, but it is still 20x the average daily gain of 0.038% (10%/year divided by 253 trading days/year).

Yet, some rare day traders can use that daily randomness to make money. I surely want to watch them in action to see how they can divine the market movement on such short terms.
 
NW-Bound said:
... Yet, some rare day traders can use that daily randomness to make money. I surely want to watch them in action to see how they can divine the market movement on such short terms.
Click to expand...
In 2018 when I was researching for my Adult-Ed investing course I spent some time with a TDAmeritrade branch manager. At that point in time they seemed to be working to become more like a Schwab or a Fido, deemphasizing their hisorical strength as a broker for retail day traders.

I asked her how the TDAmeritrade day traders had done on average the previous year (2017). There was kind of an embarassed pause and she finally said: "One and a half percent." Market index gains in 2017 ranged from 20% to almost 40%.
 
I think there are some distinctions to be made.

What do we mean by random? Implicitly, we usually mean having an uncertain outcome yet following some probability distribution. The layperson just thinks that the outcome is unknown until it is observed, yet an initial step to modeling the random outcome is to identify the appropriate probability distribution. There are many more than the Gaussian distribution.

There is abundant data out there...what isn't out there is the model, or function that reliably predicts the next day's stock price.

Consider statistical mechanics, the branch of physics that models millions of uncertain movements. It is successful because the physicists identified the model (formulae) that is consistent with the data.

What makes modeling markets so much more difficult is that the modeler is almost certainly a market participant, creating a feedback loop, and lots of modelers are at work. It's as if all the molecules in a substance are each trying to predict the flow of heat in the substance, while trading heat units with each other, according to each individual's (almost unique) predictive model.

Add to this that each individual's model is dynamic (that it changes over time, usually based on incomplete information) and the complexity of the price discovery system is extraordinary.

The below dissertation points out some of the complexity issues from a regulatory viewpoint.

https://www.cftc.gov/sites/default/...documents/file/oce_assessingfinancial1213.pdf

And to highlight, a quote
"However, as markets are continually adapting and changing, these linkages are non-constant and, thus, can create an overwhelming level of combinatorial complexity which maybe too difficult to overcome to permit effective use of traditional data analysis as a means to gain a true understanding."

So not only is the market random (tomorrow's outcome is unknown) the complexity level is beyond what we can reliably model. If someone does come up with a reliable model, it will most certainly be used in the market, and then all the other market participants will adjust their model to the knowledge of this actor.

I'd say the fanaticism of some index investors isn't necessarily based in a belief *in* indexing, but rather a skepticism that a reliable model can be hidden well enough to prevent other market participants from piggy backing on it, and thus having its advantage diluted throughout the market.
 
You must log in or register to reply here.

Similar threads

B
How often do strangers ask you for money?
3 4 5
Replies
115
Views
5K
WWDog
W
target2019
PINK etf - Investing in the Future of Health Care with Mike Taylor
Replies
4
Views
512
target2019
target2019
Brook2
Where did this investment advice come from?
Replies
2
Views
599
donheff
donheff
Chuckanut
Adjustable dumbbells
Replies
21
Views
907
2HOTinPHX
2HOTinPHX
aja8888
Prime 30 second ads..worthless
Replies
8
Views
351
Spock
S

Latest posts

Back
Top Bottom