mostlygeek / llama-swap

Model swapping for llama.cpp (or any local OpenAI API compatible server)

Repository from Github https://github.commostlygeek/llama-swap

mostlygeek/llama-swap Issues

List in llama.cpp readme
Closed 8 months ago1
[Bug/Incompatibility] Continue.dev autocompletion is broken
Closed 9 months ago3
Return errors as chat completion instead of HTTP errors
Closed 9 months ago1
Support the v1/embedding endpoint
Closed 9 months ago
Some observations and a question about error messages
Closed 9 months ago3
problems unloading the model
Closed 9 months ago3
Add Homebrew Support for macOS
Updated 9 months ago2
Support routing to multiple backends
Closed 9 months ago1
[Bug] TTL starts counting from the beginning of a request instead of the end
Closed 9 months ago2
No 'Access-Control-Allow-Origin' header present on the '/v1/models' endpoint response causes the browser to block the request due to CORS policy
Closed 9 months ago3
[Feature] Queue requests
Closed 9 months ago5
Change profile model name to use a : (colon)
Closed 9 months ago
[Feature] Support llama.cpp cache_prompt parameter
Closed 10 months ago3
Proxy does not set content length.
Closed 10 months ago5
[enhancement] Automatically unload idle model
Closed 10 months ago5
Support more API end points.
Closed 10 months ago3
Handle crashing child processes better
Closed 10 months ago
make more robust against <defunct> and crashing child processes
Closed a year ago1
Container execution
Closed 10 months ago7