PygmalionAI / aphrodite-engine

PygmalionAI's large-scale inference engine

https://pygmalion.chat

PygmalionAI/aphrodite-engine Issues

Bigger VRAM footprint after update?
Closed 5 months ago3
Reduced performance due to Ray process core pinning
Closed 5 months ago2
Error when `top_logprobs` value is `-inf`
Updated 5 months ago3
Request: Have Dockerfile use the current branch
Updated 6 months ago2
benchmark against vllm?
Updated 6 months ago2
Request: Support CLI option for concurrent request rate limit
Updated 6 months ago2
Inference results from vLLM is inconsistant with HF
Updated 7 months ago2
ModuleNotFoundError: No module named 'aphrodite.common.logits'
Closed 7 months ago5
Classifier-Free Guidance support
Updated 7 months ago3
Set ooba API Key as argument
Closed 7 months ago5
Request: Better CLI control over max CTX and rope scaling
Closed 7 months ago1
CUDA illegal memory access when loading 70b AWQ with RoPE
Closed 7 months ago8
Overcomplicated and unexplained usage for beginners
Updated 7 months ago10
GPTQRowParallelLinear has no attribute world_size
Closed 7 months ago2
Possible circular import issue
Closed 8 months ago2
"RuntimeError: CUDA error: no kernel image is available for execution on the device" on some cloud configurations
Closed 8 months ago1
PyTorch 2.1
Closed 8 months ago2
Exception in TFS implementation on simultaneous requests
Closed 8 months ago3
Error running Chat API server
Closed 8 months ago2
Pygmalion-2 generated unexpected responses
Closed 8 months ago3
Rename geppeto to GetPortaPros
Closed 8 months ago1
No logits_processors in sampling params
Closed 8 months ago4
Pygmalion6b generate unexpected texts
Closed 9 months ago2
AttributeError: 'NoneType' object has no attribute 'fs' at fresh install
Closed 10 months ago8
'activation_ops' circular import
Closed 10 months ago3