Join Early Retirement Today
Reply
 
Thread Tools Search this Thread Display Modes
Yahoo’s new interface
Old 06-08-2017, 01:12 AM   #1
Recycles dryer sheets
 
Join Date: Jul 2016
Location: North East
Posts: 172
Yahoo’s new interface

Yahoo Finance changed the way they present their data about a month ago.

I’m looking for another computer geek to help me with a problem that caused.

The ability to ask for the current prices on a list of symbols, copy the screen display, paste it inside an excel spreadsheet, and then extract the data is gone. The copy no longer contains the symbols for each requested stock or fund.

There are many places on the web where you can ask what symbols are performing the best, “TODAY”! I also wanted to know what were the best symbols a month ago, 2 months ago, etc. So that I could look for trends.


So, as a computer geek, I wrote software that used yahoo finance to get 5 years of historical market data for a list of symbols starting on any date that I wanted to see. What did the market look like on Jan 1 2017?

I can still do this on-line with a web browser, but it no longer works inside my computer program.

Does anyone know of another source for historical data?
__________________

Sandy & Shirley is offline   Reply With Quote
Join the #1 Early Retirement and Financial Independence Forum Today - It's Totally Free!

Are you planning to be financially independent as early as possible so you can live life on your own terms? Discuss successful investing strategies, asset allocation models, tax strategies and other related topics in our online forum community. Our members range from young folks just starting their journey to financial independence, military retirees and even multimillionaires. No matter where you fit in you'll find that Early-Retirement.org is a great community to join. Best of all it's totally FREE!

You are currently viewing our boards as a guest so you have limited access to our community. Please take the time to register and you will gain a lot of great new features including; the ability to participate in discussions, network with our members, see fewer ads, upload photographs, create a retirement blog, send private messages and so much, much more!

Old 06-08-2017, 09:17 AM   #2
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
Lsbcal's Avatar
 
Join Date: May 2006
Location: west coast, hi there!
Posts: 6,600
One easy solution to your problem. Find better investing techniques. Sorry, I know this is not what your were looking for.

Have you benchmarked your technique versus other easy to manage ones?
__________________

Lsbcal is offline   Reply With Quote
Old 06-08-2017, 11:19 AM   #3
Full time employment: Posting here.
Birdie Num Nums's Avatar
 
Join Date: Mar 2010
Location: Seattle-ish
Posts: 830
As a related aside, virtually every time I visit Yahoo News I get this fake "critical Mozilla Firefox patch update." I immediately close that pop-up window. And the other day I also got a fake Zeus malware warning ("do not close window or shut down your computer" warning, but call this phone number for a "fix"). Google that!
Birdie Num Nums is offline   Reply With Quote
Old 06-08-2017, 12:43 PM   #4
Thinks s/he gets paid by the post
 
Join Date: Mar 2017
Location: New York City
Posts: 2,838
i find yahoo finance to be inaccurate. they havent updated the adjusted closing price in a long time
__________________
Withdrawal Rate currently zero, Pension 137 % of our spending, Wasted 5 years of my prime working extra for a safe withdrawal rate. I can live like a King for a year, or a Prince for the rest of my life. I will stay on topic, I will stay on topic, I will stay on topic
Blue Collar Guy is offline   Reply With Quote
Old 06-08-2017, 03:38 PM   #5
Recycles dryer sheets
 
Join Date: Jul 2016
Location: North East
Posts: 172
Quote:
Originally Posted by Lsbcal View Post
One easy solution to your problem. Find better investing techniques. Sorry, I know this is not what your were looking for.

Have you benchmarked your technique versus other easy to manage ones?
Yes I have benchmarked my “investing technique”. I have been using it a little over 3 years now and over any given 12 month time frame my results have been between 6% to 10% higher than the yearly average ROR of the 3 major market indexes. I do realize that those indexes do not include things like dividends, but that only accounts for about 3% of my advantage. So, basically I am consistently doing better than the market.


Do I make mistakes, definitely, but I cash in on trends far more than losing. I just need accurate historical data to feed my software.
Sandy & Shirley is offline   Reply With Quote
Old 06-08-2017, 03:58 PM   #6
Thinks s/he gets paid by the post
 
Join Date: Nov 2011
Posts: 3,116
Is it historical price data you want? Investopedia has it, such as this link for AT&T:

http://www.investopedia.com/markets/stocks/t/
GrayHare is online now   Reply With Quote
Old 06-08-2017, 04:36 PM   #7
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
Lsbcal's Avatar
 
Join Date: May 2006
Location: west coast, hi there!
Posts: 6,600
Quote:
Originally Posted by Sandy & Shirley View Post
Yes I have benchmarked my “investing technique”. I have been using it a little over 3 years now and over any given 12 month time frame my results have been between 6% to 10% higher than the yearly average ROR of the 3 major market indexes. I do realize that those indexes do not include things like dividends, but that only accounts for about 3% of my advantage. So, basically I am consistently doing better than the market.


Do I make mistakes, definitely, but I cash in on trends far more than losing. I just need accurate historical data to feed my software.
How about looking into the data source Yahoo uses? You would probably have to pay. Once I looked at their site and they had various subscriptions. I think Yahoo has them listed in a footnote to some data, maybe in the historical data section.

You could call up Schwab. They have some of their own software and must have data too.
Lsbcal is offline   Reply With Quote
Old 06-25-2017, 02:44 PM   #8
Recycles dryer sheets
 
Join Date: Jul 2016
Location: North East
Posts: 172
I found the answer. The CSV data is no longer accessible for computer program screen scraping, but the standard HTML format can still be used.

This post is for the total geeks who are trying to screen scrape historical data. If you are not a total geek you are probably scratching your head already and asking what the heck is screen scraping!

Yahoo now uses time_t to represent the starting and ending dates for historical data. Time_t is the number of seconds since 1/1/1970 represented as a 32 bit signed integer. It will reach its max on January 19, 2038, so we still have time to use it.

The yahoo finance URL for getting the 37 monthly adjusted closing prices for DNP from May 1, 2014 to Jun 1, 2017 is:

https://finance.yahoo.com/quote/DNP/history?period1=1398916800&period2=1496289600&inte rval=1mo&filter=history&frequency=1mo


It uses time_t values for the date range and sets the interval and frequency for monthly data.

OK, if you are not a total geek you will now get totally lost.

Scan the HTML code of the returned page for “Currency in USD”, then search for each table row TR in the first TABLE that follows that string.

Remember the content of the first column in each row as the date. If there is a TD 6th column /TD, then this is a historical adjusted closing price for that date. If there is not a 6th column the row is only telling you the dividend or split information.

If you are a total geek and are interested in the code that I wrote, send me a PM and we can talk more about it. If you are not a total geek you probably didn't read to the end of this post anyway!
Sandy & Shirley is offline   Reply With Quote
Old 06-25-2017, 03:35 PM   #9
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
Lsbcal's Avatar
 
Join Date: May 2006
Location: west coast, hi there!
Posts: 6,600
Quote:
Originally Posted by Sandy & Shirley View Post
I found the answer. The CSV data is no longer accessible for computer program screen scraping, but the standard HTML format can still be used.
....
I am a little confused. I use Yahoo historical to get monthly performance data. When I download a series it comes across as a .csv file. The numbers seem to be consistent recently. I checked that the monthly returns for VINEX are the same as my past data for the last 7 years.

So if by "screen scraping" you mean taking the "Adj. Close" prices to compute the monthly returns, then that still seems to be fine. Note that the numbers are adjusted for new dividends. So all the numbers in a series are backward adjusted on the new dividend month. But the differentials (to compute the monthly return) should remain the same.
Lsbcal is offline   Reply With Quote
Old 06-26-2017, 01:27 AM   #10
Recycles dryer sheets
 
Join Date: Jul 2016
Location: North East
Posts: 172
What I’m talking about, geek speak, is the difference between using a browser like IE or FF to look at the data with your eyes, and in this case click on a link to download the data and get a popup screen asking how you want the data sent back to you. That browser has been keeping special protocol data, like cookies (geek speak), which it can pass back to yahoo so that they know that the browser is authorized to receive the requested CSV file.

If I understand you, you do this one symbol at a time, open the returned file in excel, and then do your analysis.

The software I wrote is following 355 symbols at the same time, based on the list of Fidelity no transaction fee funds. At the end of the trading day, maybe 7 PM after all of the funds have posted their closing prices, I click a single link on my desktop. A “command prompt” window opens up and begins to display today’s data. It take 2 to 3 seconds per symbol and finishes in about 10 minutes at which time the widow makes a series of quick beeping sounds, waits 10 seconds just in case I want to look at it for some reason, then closes on its own.

I then run my own “Market View” software that can look at a snapshot, calculate the monthly, quarterly, and annual RORs for each symbol and display which symbols did the best for each of those time periods. It also allows me to create custom “views” of the market where I can ask which symbols did the best when looking at multiple RORs at the same time, show me the best 1 year returns that are still doing well over the last 3 months.

My snapshot program can also be run for historical dates, what the market looked like on March 31, 2017. I keep monthly historical snapshots for the past 5 years. All of the data that I can find on line lets me look at what the best investments are “today”. I can analyze the latest snapshot and also get the same information, but I can then ask what was the best investment a month ago, the month before that, or last year, etc.

All of this is only possible because I can look at 355 symbols on any historical date in about 10 minutes or less. I have 5 years of monthly snapshots saved on my disk. My custom market view software loads all of this data and computes all of the RORs and Rankings in less than 5 seconds so that I can then view the data instantly in multiple formats.

This is only possible because of the snapshot process that can “look at” 355 web pages in 10 minutes, extract (screen scrape) the data I want and save it on my disk.

That program used to be able to ask for the CSV data directly from yahoo, but they changed their protocol for requesting that data so all of my personal market analysis software came to a screeching halt. I now have to analyze the background HTML to extract that data. If you want to see what I am talking about, display any historical view page on your browser, right click on the page and select “view page source”. That is what the HTML looks like that displayed the humanly readable page on your browser. That is what a screen scraping process looks at one character at a time, to find and save the data you want.
Sandy & Shirley is offline   Reply With Quote
Old 06-26-2017, 08:14 AM   #11
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
Lsbcal's Avatar
 
Join Date: May 2006
Location: west coast, hi there!
Posts: 6,600
OK, thanks for the overview. Didn't realize you were looking at such a large set of funds. My needs are much more modest and for equities I only need about 10 fund monthly updates.
__________________

Lsbcal is offline   Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Yahoo finance new format is useless. Sandy & Shirley Other topics 47 06-23-2017 03:24 PM
Yahoo Finance - Dead is the new RE! DougViages Life after FIRE 47 07-06-2009 11:54 AM
Yahoo Finance's new charting feature. Ed_The_Gypsy FIRE and Money 4 12-29-2006 12:00 PM

» Quick Links

 
All times are GMT -6. The time now is 12:52 PM.
 
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2019, vBulletin Solutions, Inc.