i added hugging face read api key, but when i try to run a query i get a 503

Question

i added hugging face read api key, but when i try to run a query i get a 503

ralyodio opened this issue a year ago · comments

ralyodio · Answer 1 · Tue Apr 25 2023 14:38:54 GMT+0800 (China Standard Time)

INFO:server.lib.inference:Requesting inference from databricks/dolly-v2-12b on huggingface
INFO:werkzeug:127.0.0.1 - - [24/Apr/2023 23:37:51] "POST /api/inference/text/stream HTTP/1.1" 200 -
ERROR:server.lib.inference:Error: Request failed: 503 Service Unavailable

ralyodio · Answer 2 · Tue Apr 25 2023 14:39:03 GMT+0800 (China Standard Time)

how do i fix?

Zain Huda · Answer 3 · Tue May 02 2023 04:42:24 GMT+0800 (China Standard Time)

seems like this is the huggingface API, you'll have to wait a bit for the model to "warm up" before you can call for inference. https://huggingface.co/docs/api-inference/faq#:~:text=Rate%20limits&text=We%20try%20to%20balance%20the,errors%20saying%20models%20are%20loading.