pydata / pandas-datareader

Extract data from a wide range of Internet sources into a pandas DataFrame.

Home Page:https://pydata.github.io/pandas-datareader/stable/index.html

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

can not fetch data before 1970 on yahoo finance

fenixnano opened this issue · comments

Hi!
I run into an error when using pandas-datareader. I think it's probably related to the unix time stamp.

my code:

from pandas_datareader import data as web
import datetime
import pandas as pd
import matplotlib.pyplot as plt
#spy 500 index
stock = '%5EGSPC'
endDate = datetime.datetime(2022, 8, 30)
start_date = datetime.datetime(1928, 12, 29)
df = web.DataReader(stock, data_source='yahoo', start=start_date,end=endDate)'

and I get error:

'envs\stock_predict\lib\site-packages\pandas_datareader\yahoo\daily.py in _get_params(self, symbol)
124 # This needed because yahoo returns data shifted by 4 hours ago.
125 four_hours_in_seconds = 14400
--> 126 unix_start = int(time.mktime(self.start.timetuple()))
127 unix_start += four_hours_in_seconds
128 day_end = self.end.replace(hour=23, minute=59, second=59)

OverflowError: mktime argument out of range'

I tried and it works for a time after 1970, so seems it's due to converting of the UNIX time stamp.

Could you check it? Thanks!

I came across the same issue on Fred's data and researched.

Check '4.16 Seconds Since the Epoch'. There is no standard if unit time became negative.
https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap04.html

I feel it's too hard to reverse engineer since lot's users or vendors defined conversion formula.