FlurlHttpException with StatusCode: 429, ReasonPhrase: 'Too Many Requests'
irigopou opened this issue · comments
Since last night, prices cannot be retrieved from Yahoo Finance.
The failure occurs in Yahoo - Quote.cs in the following method
public async Task<IReadOnlyDictionary<string, Security>> QueryAsync(CancellationToken token = default)
Below, the FlurlHttpException ex is caught.
try
{
data = await url
.WithCookie(YahooSession.Cookie.Name, YahooSession.Cookie.Value)
.GetAsync(token)
.ReceiveJson()
.ConfigureAwait(false);
}
catch (FlurlHttpException ex)
The message in ex contains:
ResponseMessage = {StatusCode: 429, ReasonPhrase: 'Too Many Requests'
Can confirm the same. Possible rate limits on the part of Yahoo, but why all of a sudden.
Rate limit? Definitely not, as the exemption was thrown immediately after the first request.
I am hitting the same too.
I have found a way. Tip came from here:
https://stackoverflow.com/questions/78111453/yahoo-finance-api-file-get-contents-429-too-many-requests
Do the same thing as in InitAsync() when we get the crumb:
https://github.com/karlwancl/YahooFinanceApi/blob/master/YahooFinanceApi/YahooSession.cs
Which was:
const string userAgentKey = "User-Agent";
const string userAgentValue = "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36";
...
.WithHeader(userAgentKey, userAgentValue)
.GetAsync()
So, both in Historical.cs and in Quote.cs, put this line before the GetAsync()
.WithHeader("User-Agent", "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36")
.GetAsync(...
Works for me at the moment. (from the UK).
Somebody could clean the quick-and-dirty solution and create a PR to fix it properly in the repo.
One proper solution is to create the string userAgentKey/userAgentValue as data member fields somewhere, and all 3 GetAsync() could use that.
Feeds work again now, even without the change proposed above!
Thx for checking.
Maybe it works for you now without the proposed changes, but you might use it from a different country (USA?). I am from London.
And I just checked it again right now. 5 minutes ago.
For me, this line is still needed at this moment. Otherwise I get the Furl exception.
.WithHeader("User-Agent", "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36")
Let's hear the experiences of others.
I am in Germany. It started failing last night, but it worked again about an hour ago.
I made code changes as suggested. Please try it.
It works from USA today with and without the changes.
However, there are random 401 errors are occurring during the testing.
Funny enough. 30min after my previous post (that I double-checked), now in the UK, I get the same as you:
It started working with or without the changes. Hmmm.
I also have the random 'Call failed with status code 401 (Unauthorized)' when we get the Crumb.
But usually, it works after the second try.
Thx for sharing the experience.
In AU I too get some random 401
Call failed with status code 401 (Unauthorized): GET https://query1.finance.yahoo.com/v1/test/getcrumb