Add TimeZone Support For Historical Apis
karlwancl opened this issue · comments
I'm still having some issues. Running the below line is returning me 2 records
await Yahoo.GetHistoricalAsync("rxp.ax", new DateTime(2017,10,8), new DateTime(2017,10,9), Period.Daily, timeZone: "AUS Eastern Standard Time");
But the one for today (2017/10/09) has different values than the Yahoo website and the one for yesterday (2017/10/08) has the correct values for today. However there should be no records for the 8th as that was a sunday.
@ntrous The result shown on Yahoo's web page does not match with the result shown on Yahoo's csv, thus the incorrect datetime on the lib. The latest version should fix the issue. Please try it to see if it works :)
Now I'm getting a record for tomorrow, so it seems to have shifted a day in the wrong direction.
I don't think this library should be involved with timezones. I suggest that the date arguments be the caller's responsibility.
@dshe That's fine, it's just as the caller I have no idea what timezone the data is going off. It seems local time to the exchange is not what yahoo is using. Perhaps they store all their data as UTC?
I believe that the dates sent to Yahoo Finance are interpreted as the number of seconds after Jan 1, 1970 at the particular exchange.
I have made some modifications to my fork. Your feedback would be appreciated:
@ntrous , @dshe has mentioned it correctly, yahoo get their datetime from number of seconds from 1970/1/1 00:00:00 UTC.
However, the result shown on the site involves timezone, and may not be exactly the same as in csv version which the library is parsing from.
For instance, on web, if I am querying from 2016/1/1 to 2016/1/2, the query datetime would be recognized as 2016/1/1 (your timezone) to 2016/1/2 (your timezone), and the datetime of the result is also translated to your timezone.
But for the csv version, it remains in UTC, i.e. you are querying from 2016/1/1 (your timezone) to 2016/1/2 (your timezone), and the result is NOT translated to your timezone, that's the problem with the csv version (the lib uses).
@dshe has solved the issue on his PR on unifying the input & output datetime by using utc datetime, it's a perfect solution to the problem but i am wondering there may be a potential break if user upgrades their lib. Some records may not be included in the utc datetime range which is included in the local datetime range before, that's what i'm hesitating.
How about @dshe creates an overload for those functions and you run both versions for a period of time? Mark the current method as Deprecated and that way existing users of the lib can leave it as is, until they migrate their code.
I did some further testing and now believe that my date calculations were incorrect. I have made changes and revised my pull request.
I suggest you pre-release a version 2.0 with the breaking changes.
@dshe Thank you for your PR, I need some time to review the code, i will release the pre-release to nuget once i have completed 😄
@ntrous I have just pushed the beta version to nuget, please update it accordingly & help testing it, thanks :)
@lppkarl Thank you. I updated my fork and it seems fine.
I don't think there is any value in using Matt Johnson's TZConvert.GetTimeZoneInfo("Eastern Standard Time"), rather than simply calling System.TimeZoneInfo.FindSystemTimeZoneById("Eastern Standard Time").
@dshe TZConvert is used here because non-Windows platform doesn't use Windows TimeZone Id. For me, as a mac user, the FindSystemTimeZoneById statement simply throws timezone not found exception. The TZConvert is in replacement here to provide abstraction on top of the underlying timezone system (windows/macOS/linux, etc) so that i do not need to worry about the OS of the lib users, hopes you could understand this 😅
@lppkarl Thanks for your explanation. I could be wrong but I think FindSystemTimeZoneById() should work everywhere since it is supported by NetStandard. If it doesn't work on Mac, that would be a bug. Futhermore, Matt Johnson's library itself calls FindSystemTimeZoneById() on line 72 of TZConvert.cs.
@dshe you are right that the FindSystemTimeZoneById() itself works on every platform that implements netstandard, but what makes it non-workable is that windows & macOS/linux uses different sets of timezone id to represent the same timezone.
E.g. For windows, you use "Eastern Standard Time" to represent the timezone GMT-5. However, there's nothing called "Eastern Standard Time" in macOS/linux but there is "America/New_York" that represents the same thing.
That's the main reason to use an external library to implicitly convert between them. But I am thinking of replacing it with simple if/else by using System.Runtime.InteropServices.RuntimeInformation.IsOSPlatform() to determine the underlying OS
It looks to work better. For whatever reason the yahoo csv file thinks that todays date is yesterdays date.
For instance, if I query RXP.AX on the yahoo website, the most recent record in the csv file is for 17/10/2017 but the numbers line up to the values for today 18/10/2017.
Not a fault of this library obviously, but something I'll have to account for in my application.
@lppkarl You are right. The timezone Ids are not consistent across platforms. So I suggest this:
private static readonly TimeZoneInfo TzEst = TimeZoneInfo
.GetSystemTimeZones()
.Where(tz => tz.Id == "Eastern Standard Time" || tz.Id == "America/New_York")
.SingleOrDefault();
@lppkarl I made a new PR for the timezone issue. But for #17, I was hoping that you could update the dependencies because it would not be easy for me.
@ntrous I see the problem you noted but I am not sure what to do about it. The YahooFinanceApi library just returns the dates it receives from Yahoo.
Yeh the problem for me is fixed so close the issue. Thanks for all the work guys!