akashsonowal / ml-server

A high throughput and low latency machine learning server

ml-server

A high throughput and fast machine learning server

git clone https://github.com/akashsonowal/ml-server.git
cd ml-server
pip install -e .

Examples

cd examples

# text generation
python text_generator.py

Benchmarks

cd benchmarks
locust