ray-project / llmperf

LLMPerf is a library for validating and benchmarking LLMs

ray-project/llmperf Issues

unable to get the benchmark result
Updated a month ago1
Blocking on pending requests despite block == false
Updated 2 months ago
HUGGINGFACE set
Updated 3 months ago
bug of counting output tokens
Updated 3 months ago
llmperf not working for concurrent users
Updated 4 months ago
Bug: Hugging Face TGI not working
Updated 4 months ago
Concurrency level is not handled properly
Updated 4 months ago
Add memory bandwidth utilization metric
Updated 4 months ago
Handling custom codes arguments like trust_remote_code
Updated 5 months ago
usage for local models .
Updated 5 months ago
Basic usage issue
Updated 5 months ago
Are benchmark results released somewhere
Closed 5 months ago5
litellm serializable issue ?
Updated 6 months ago
Crashing with large number of concurrent users
Updated 6 months ago
Does llmperf support measuring local disk models? What's the meaning of framework in llmperf.py line 355?
Closed 7 months ago3