NVIDIA / trt-llm-as-openai-windows

This reference can be used with any existing OpenAI integrated apps to run with TRT-LLM inference locally on GeForce GPU on Windows instead of cloud.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

NVIDIA/trt-llm-as-openai-windows Stargazers