Nothing displayed - Yahoo API breaking contract change

Question

Nothing displayed - Yahoo API breaking contract change

clauseggers opened this issue a year ago · comments

Claus Eggers Sørensen commented a year ago

When I open ticker I get no stocks listed. Same as in #246 so perhaps they are related.
I’m on version 4.5.6.
I tried opening without the config file, but got same result.
I can run something like curl https://query1.finance.yahoo.com/v6/finance/quote\?symbols\=AAPL,GOOG,F | jq and get good output.

Running on MacOS 13.3.1 (a), installed with Ticker was installed with Homebrew.

hchr · Answer 1 · Fri May 05 2023 19:23:12 GMT+0800 (China Standard Time)

v7 strikes again it seems
curl https://query1.finance.yahoo.com/v7/finance/quote?symbols=tsla
{"finance":{"result":null,"error":{"code":"Unauthorized","description":"Invalid Cookie"}}}13:14

Matt Radford · Answer 2 · Fri May 05 2023 22:50:04 GMT+0800 (China Standard Time)

Rebuilding ticker after search/replacing v7 with v6 seems to fix the issue.

Is there interest in making the yahoo API version configurable?

Changhee Oh · Answer 3 · Sat May 06 2023 05:58:58 GMT+0800 (China Standard Time)

Same issue.
It seems to be related to macOS latest update.

OS:

ProductName: macOS
ProductVersion: 13.3.1

Ticker version: 4.5.6

Robert James Hernandez · Answer 4 · Sat May 06 2023 06:07:17 GMT+0800 (China Standard Time)

Its broken for me on linux too. No recent OS updates have been applied for me:

$ ticker -v
ticker version v4.5.5

I suspect its a similar outage to a few weeks ago: #246

Ani Channarasappa · Answer 5 · Sat May 06 2023 07:15:33 GMT+0800 (China Standard Time)

Thanks for all the reports! I've created a new release to use the v6 Yahoo API which should fix it but I'll leave this open for a bit to confirm

ajgringo619 · Answer 6 · Sat May 06 2023 07:51:54 GMT+0800 (China Standard Time)

Working as advertised here - thanks for the quick fix!

Ani Channarasappa · Answer 7 · Mon May 08 2023 21:41:27 GMT+0800 (China Standard Time)

Try with the latest version 4.5.10

Longer term some kind of abstraction layer should help mitigate these sorts of issues in the future.

Claus Eggers Sørensen · Answer 8 · Wed May 24 2023 17:43:01 GMT+0800 (China Standard Time)

Did Yahoo just shut off the v6 API?

v6

curl https://query1.finance.yahoo.com/v6/finance/quote\?symbols\=AAPL,GOOG,F | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    91  100    91    0     0   1489      0 --:--:-- --:--:-- --:--:--  1625
{
  "finance": {
    "result": null,
    "error": {
      "code": "Not Found",
      "description": "HTTP 404 Not Found"
    }
  }
}

v7

curl https://query1.finance.yahoo.com/v7/finance/quote\?symbols\=AAPL,GOOG,F | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    90  100    90    0     0   1722      0 --:--:-- --:--:-- --:--:--  1914
{
  "finance": {
    "result": null,
    "error": {
      "code": "Unauthorized",
      "description": "Invalid Cookie"
    }
  }
}

Valentin Vidić · Answer 9 · Thu May 25 2023 02:39:42 GMT+0800 (China Standard Time)

Probably the same problem described here:

https://stackoverflow.com/questions/76065035/yahoo-finance-v7-api-now-requiring-cookies-python

Ani Channarasappa · Answer 10 · Thu May 25 2023 10:50:58 GMT+0800 (China Standard Time)

Thanks for reporting this - I can replicate the issue but there is no simple workaround as before so it may take some time to fix this issue.

@vvidic thanks for sharing the above link. I was not able fix the issue by following the steps listed in the link but I'll keep searching to see if someone else has found a workaround.

Ani Channarasappa · Answer 11 · Mon May 29 2023 11:33:38 GMT+0800 (China Standard Time)

I've made an update that should restore functionality with v4.5.13. I'll close this issue for now but please re-open if this happens again.

For those interested, there are three differences with the v7 Yahoo API in the latest breaking change:

API calls must have the A3 cookie set
a. Sessions expire after about 6 hours
b. Cookies can be retrieved by visiting the Yahoo Finance homepage (example)
API calls must have the URL query parameter crumb set
a. "crumbs" can be retrieved by requesting GET https://query2.finance.yahoo.com/v1/test/getcrumb (example)
API contract for the response from GET v7/finance/quote has changed for several fields to include both raw and formatted values (example)

Claus Eggers Sørensen · Answer 12 · Mon May 29 2023 15:12:17 GMT+0800 (China Standard Time)

Thank you @achannarasappa but I can’t get past executing the app. When I press return I just get the cursor moved to the next line, where it is then stuck seemingly doing nothing. In some terminal emulators it blinks a few times before going solid.

hchr · Answer 13 · Mon May 29 2023 20:32:22 GMT+0800 (China Standard Time)

Thank you @achannarasappa but I can’t get past executing the app. When I press return I just get the cursor moved to the next line, where it is then stuck seemingly doing nothing. In some terminal emulators it blinks a few times before going solid.

I get the same here on Arch
ticker -v
ticker version v4.5.13

Added:
It is connected and the data is flowing in:
Sent Recived
ticker eno1 7.831 38.404 KB/sec

works for some it seems.
Maybe OS related?

jfbourdeau · Answer 14 · Mon May 29 2023 20:38:23 GMT+0800 (China Standard Time)

Tks for the last update. Everything is working again. JF MTL Canada

Ani Channarasappa · Answer 15 · Mon May 29 2023 23:59:23 GMT+0800 (China Standard Time)

Looks like there's still an issue but it does not impact everyone. It could be region or machine specific so for those that have issue, would you mind sharing country and machine details to help pinpoint the issue?

Claus Eggers Sørensen · Answer 16 · Tue May 30 2023 00:05:52 GMT+0800 (China Standard Time)

I’m on MacOS latest, installed ticker with Homebrew, and I’m located in the Netherlands.

097115 · Answer 17 · Tue May 30 2023 00:18:57 GMT+0800 (China Standard Time)

Ubuntu 22.04.2, downloaded the binary from the Releases page, another EU country. Same symptoms as described above.

hchr · Answer 18 · Tue May 30 2023 03:25:41 GMT+0800 (China Standard Time)

System:
Kernel: 6.3.4-arch1-1
Desktop: KDE Plasma v: 5.27.5
Distro: EndeavourOS
CPU:
model: AMD Ryzen 9 5900HX with Radeon Graphics

Denmark

Kjell · Answer 19 · Tue May 30 2023 04:46:41 GMT+0800 (China Standard Time)

I implemented the same approach in my own app and it works now. However it does not work with "finance.yahoo.com" in the initial request to get the cookie, I have to use "fc.yahoo.com".

eriknh · Answer 20 · Tue May 30 2023 05:37:21 GMT+0800 (China Standard Time)

I just updated via the snap updater on Ubuntu server 22.04 to ticker version 4.5.13 and the app works as expected again. However I did not set any cookies. How is it working without me setting a cookie or specifying the CRUMB value?

Also, can I set these somehow as a variable upon calling the "ticker" command? How would I input my own cookie variables otherwise?

Thank you for the update @achannarasappa and thank you for the explanations.

Kjell · Answer 21 · Tue May 30 2023 19:30:36 GMT+0800 (China Standard Time)

You don't have to set anything, the code does it.

Cubuss · Answer 22 · Tue May 30 2023 20:45:27 GMT+0800 (China Standard Time)

i get nothing using snap ticker version 4.5.13

ticker -w GME it just hangs still no output

eriknh · Answer 23 · Tue May 30 2023 23:25:53 GMT+0800 (China Standard Time)

You don't have to set anything, the code does it.

Thanks @ktpx! Great to know the code does this for us. @achannarasappa amazing development on that CRUMB/COOKIE effort. Would love to know if any of these Yahoo Finance variables gets added to the config file should Yahoo start getting finicky about access.

Ani Channarasappa · Answer 24 · Wed May 31 2023 10:25:17 GMT+0800 (China Standard Time)

I was able to replicate the issue reported by EU users. On finance.yahoo.com, there is a prompt related to EU data privacy regulations that the user must accept before using the Yahoo Finance. Without accepting this prompt, the required cookie (A3) is not set. ticker v4.5.13 does not support this today. Look out for an update here soon that adds support.

Kjell · Answer 25 · Wed May 31 2023 17:02:46 GMT+0800 (China Standard Time)

I was able to replicate the issue reported by EU users. On finance.yahoo.com, there is a prompt related to EU data privacy regulations that the user must accept before using the Yahoo Finance. Without accepting this prompt, the required cookie (A3) is not set. ticker v4.5.13 does not support this today. Look out for an update here soon that adds support.

Thats probably why fc.yahoo.com works here (im in eu), and not finance.yahoo.com.

Ani Channarasappa · Answer 26 · Wed Jun 07 2023 09:55:55 GMT+0800 (China Standard Time)

v4.5.14 has just been released which fixes the quote retrieval issue for EU users. Closing this issue for now but will reopen if there's still any unresolved problems.

Thanks all for providing details and reports that helped understand the issue better!