Amanda-Barbara / blora-text-generation-inference

Batched LORA + Continuous Batching

Geek Repo

Github PK Tool

blora-text-generation-inference

Batched LORA + Continuous Batching

About

Batched LORA + Continuous Batching

Apache License 2.0

Languages

Language:Python 65.9%Language:Rust 18.0%Language:Jupyter Notebook 9.8%Language:Cuda 4.7%Language:Dockerfile 0.6%Language:C++ 0.6%Language:Makefile 0.2%Language:JavaScript 0.2%Language:HTML 0.1%Language:C 0.0%