Giters

microsoft / batch-inference

Dynamic batching library for Deep Learning inference. Tutorials for LLM, GPT scenarios.

Home Page:https://microsoft.github.io/batch-inference/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

microsoft/batch-inference Stargazers