LlamaEdge / LlamaEdge

The easiest & fastest way to run customized and fine-tuned LLMs locally or on the edge

https://llamaedge.com/

LlamaEdge/LlamaEdge Issues

question: Using chat on Android device
Updated a month ago5
Output of llama-chat.wasm and llama-api-server.wasm is different
Updated a month ago25
Model list on `https://huggingface.co/second-state`
Updated a month ago
bug: failed to run install !
Updated a month ago4
Failed to build LlamaEdge
Closed a month ago6
question: Support embedding?
Closed a month ago5
bug: failed to allocate buffer
Updated a month ago3
Feature Request: Is it possible to see the output realtime?
Updated a month ago7
bug: Phi-2 support is not exposed via the API Server CLI
Updated 2 months ago8
feat: Support for OpenAI Function Calling (for full drop-in replacement)
Updated 2 months ago3
instantiation failed: unknown import, Code: 0x62 #60question:
Closed 2 months ago3
Feature Request: add Number of Threads for better performance
Updated 2 months ago
bug: Installation failed
Updated 2 months ago4
bug: process getting aborted on raspberrypi-5
Closed 2 months ago6
Feature Request: Improve the README.md about `run-llm.sh` section
Closed 2 months ago
Feature Request: Output response of API server
Updated 2 months ago3
Migration: Update the urls to the new repo
Updated 2 months ago1
Feature Request: Simplify the run-llm.sh script interactions
Closed 3 months ago
Language SDKs: start a WasmEdge server from my own app
Updated 3 months ago
bug: Issue with gemma
Updated 3 months ago2
Feature Request: Use `data: [DONE]` as a stop sign of streaming output
Updated 3 months ago
Feature Request: Allow cursor keys in chat (streaming mode)
Updated 3 months ago
bug:
Updated 3 months ago1
bug: When using the baichuan2-13b-chat-q4-k-m model, the output is completely nonsensical.
Updated 3 months ago4
The model is loaded into the GPU, but inference is still performed through the CPU.
Closed 3 months ago
bug: Failed to run start script
Closed 3 months ago15
Feature Request: Android support
Updated 4 months ago1
max_token not working?
Closed 4 months ago9
Feature Request: method to run llama-api-server on demand
Closed 4 months ago2
Model requests
Closed 4 months ago7
bug: called `Result::unwrap()` on an `Err` value: Utf8Error { valid_up_to: 963, error_len: None }
Updated 4 months ago1
bug: cannot detect CUDA if the user does not install nvcc
Closed 4 months ago8
bug: load library failed:libcudart.so.12: cannot open shared object file: No such file or directory
Closed 4 months ago9
bug: cannot install on RHEL 8.6
Closed 4 months ago2
Feature Request: update the model list in run-llm.sh
Closed 4 months ago2
question: Running (small) LLMs on RaspberryPI 5
Closed 4 months ago10
Can't run llama2-7b on cuda 12.1
Closed 4 months ago2
The api server should respond status code 503 when it is busy
Updated 4 months ago
Feature Request: Create a multi thread api server demo
Updated 4 months ago
bug: invalid value 'mixtral-instruct' for '--prompt-template <TEMPLATE>'
Closed 4 months ago1
Failed to run model on windows
Closed 4 months ago10
question: How to check out token per second when running a model
Closed 4 months ago8
Add Mixtral 8X7b to the models.md and run-llm.sh
Closed 4 months ago1
question: Running Gorilla OpenFunctions v1?
Closed 4 months ago1
bug: Error("control character (\\u0000-\\u001F) found while parsing a string
Closed 5 months ago3
bug: Can‘t interact Samantha-1.2-Mistral-7B with API server
Closed 5 months ago1
bug: Can't interact WizardCoder-Python-7B-V1.0 with API server
Closed 5 months ago1
bug: Can't interact Zephyr-7B-Alpha with API server
Closed 5 months ago1
bug: Can't interact Deepseek-Coder-6.7B with API server
Closed 5 months ago1
bug: Can't interact OpenHermes-2.5-Mistral-7B with API server
Closed 5 months ago