Dynamic batching library for Deep Learning inference. Tutorials for LLM, GPT scenarios.
Home Page:https://microsoft.github.io/batch-inference/
Geek Repo:Geek Repo
Github PK Tool:Github PK Tool