Need to provide additional args to InferenceClient
dylan-stark opened this issue · comments
I'm using huggingface.chatCompletion and need to be able to provide some cookies to the InferenceClient. I don't see a way to pass that in via create(), which is where the client is instantiated. I also looked at the huggingface_hub implementation, but didn't see a way to set it via module member or environment variable.
The approach I landed on was adding a module-level client_args dictionary, similar to huggingface.api_key and .prompt_builder, and passing anything defined in there when the client is instantiated. If that makes sense and you want to support this, I'm happy to open a PR. Or if you see issues or have suggestions to improve, let me know.
Hello @dylan-stark,
Currently there is no way to customize the InferenceClient.
If you want to can create a PR with a recommendation on how to add those.
Hi @philschmid. I opened PR #45. It's working well for me in practice. If there are some changes you'd recommend, I'm happy to make them.