 |
05-15-2015, 02:34 PM
|
#1
|
|
Thinks s/he gets paid by the post
Join Date: Jun 2010
Posts: 2,301
|
How backtesting fails
I thought this was a very good article (although somewhat technical) about how backtesting can fail or mislead:
9 Mistakes Quants Make that Cause Backtests to Lie by Tucker Balch, Ph.D. | Quantopian Blog
I never did any backtesting with financial/stock data (beyond running FIRECALC etc), but I saw a lot of these errors in other areas during my former work life. These were even made by people with advanced degrees in quantitative fields.
Just something to think about for those trying to interpret backtesting studies or running their own tests...
|
|
|
 |
Join the #1 Early Retirement and Financial Independence Forum Today - It's Totally Free!
Are you planning to be financially independent as early as possible so you can live life on your own terms? Discuss successful investing strategies, asset allocation models, tax strategies and other related topics in our online forum community. Our members range from young folks just starting their journey to financial independence, military retirees and even multimillionaires. No matter where you fit in you'll find that Early-Retirement.org is a great community to join. Best of all it's totally FREE!
You are currently viewing our boards as a guest so you have limited access to our community. Please take the time to register and you will gain a lot of great new features including; the ability to participate in discussions, network with our members, see fewer ads, upload photographs, create a retirement blog, send private messages and so much, much more!
|
|
05-15-2015, 03:07 PM
|
#2
|
|
Thinks s/he gets paid by the post
Join Date: Apr 2012
Location: Nashville
Posts: 2,308
|
Thanks. Interesting, and I would not have been likely to run across it.
__________________
OMY * 3 2ish Done 7.28.17
|
|
|
|
|
05-15-2015, 03:14 PM
|
#3
|
|
Thinks s/he gets paid by the post
Join Date: Jul 2009
Posts: 1,934
|
Good article and interesting website. Thanks.
__________________
And if I claim to be a wise man, it surely means that I don't know.
|
|
|
|
|
05-15-2015, 05:49 PM
|
#4
|
|
Thinks s/he gets paid by the post
Join Date: Oct 2006
Posts: 4,629
|
Interesting article, Thanks.
I was surprised by this:
" If, instead we use the members of the S&P 500 starting in 2008, we find that more than 10% of the listed companies failed."
Do you know if "failed" means "went bankrupt"? or "dropped out of the S&P 500"?
|
|
|
|
|
05-16-2015, 11:23 AM
|
#5
|
|
Thinks s/he gets paid by the post
Join Date: Jun 2010
Posts: 2,301
|
When I first read the article I thought the 10% figure was a hypothetical. But I think after some googling that the writer is referring to the number of companies removed for distress reasons (see attached figure from S&P 500 Deletions Due To Distress - Business Insider ). The removals from distress seem to match his 10% figure.
From the article
 Quote:
|
the S&P 500 deletions that were a consequence of stocks that failed outright, were removed due to substantial declines in their market value, or were acquired after suffering such a decline
|
|
|
|
|
|
05-16-2015, 11:47 AM
|
#6
|
|
Recycles dryer sheets
Join Date: Jun 2014
Posts: 440
|
That's pretty neat. I just looks at the richest lists an don't see many (any) quant traders on those lists so I figure... Can't work that well 
Sent from my HTC One_M8 using Early Retirement Forum mobile app
|
|
|
|
|
05-16-2015, 05:46 PM
|
#7
|
|
Thinks s/he gets paid by the post
Join Date: Oct 2006
Posts: 4,629
|
Quote:
Originally Posted by photoguy
When I first read the article I thought the 10% figure was a hypothetical. But I think after some googling that the writer is referring to the number of companies removed for distress reasons (see attached figure from S&P 500 Deletions Due To Distress - Business Insider ). The removals from distress seem to match his 10% figure.
From the article |
Thanks. That's a higher number than I would have guessed. Nice chart.
|
|
|
|
|
|
Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
|
|
|
| Thread Tools |
Search this Thread |
|
|
|
| Display Modes |
 Linear Mode
|
Posting Rules
|
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
HTML code is Off
|
|
|
|
» Recent Threads
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
» Quick Links
|
|
|
