Giters

theodo-group / GenossGPT

One API for all LLMs either Private or Public (Anthropic, Llama V2, GPT 3.5/4, Vertex, GPT4ALL, HuggingFace ...) 🌈🐂 Replace OpenAI GPT with any LLMs in your app with one line.

Home Page:https://genoss.ai/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Rate limit Genoss GPT.

mattzcarey opened this issue · comments

commented

Add capability for rate limiting users

commented

Hi @mattzcarey ,

I think it might be a good goal to test extending Genoss based on FastAPI plugins.

You have a stackoverflow on the subject here : https://stackoverflow.com/questions/65491184/ratelimit-in-fastapi
With a bunch of packages, for example this one : https://github.com/laurentS/slowapi .

I would test it and document how to do this rather than directly implementing it.