PotatoSpudowski / fastLLaMa

fastLLaMa: An experimental high-performance framework for running Decoder-only LLMs with 4-bit quantization in Python using a C/C++ backend.

https://potatospudowski.github.io/fastLLaMa/

PotatoSpudowski/fastLLaMa Issues

GGUF and/or LLama-3 support?
Updated 5 months ago
"No module named 'fastllama.api' " after pip installation
Updated a year ago10
how to load model in webui ?
Updated a year ago3
Webui UX issue on mobile
Updated a year ago
Port llama.cpp openCL support to fastllama?
Updated a year ago
Make running fastLLaMa on windows simple!
Updated a year ago1
README.md is outdated in sections #running-llama and #running-alpaca-lora
Updated a year ago1
Deciding the Schema for the protocol between webUI and webSocket Server
Closed a year ago2
Integrating + Testing webUI and WebSocket Server
Closed a year ago
Implement the WebSocket Server
Closed a year ago
Pip support testing
Closed a year ago21
Designing the UI
Closed a year ago1
Pip uninstall not removing the package
Updated a year ago2
How install on Windows?
Closed a year ago4
TypeError: Model.generate() got an unexpected keyword argument 'stop_word'
Closed a year ago2
Should fastLLaMa support more than just Python?
Closed a year ago3
Feature suggestions!
Closed a year ago7
Enabling custom logger makes it crash at ingestion.
Closed a year ago1
from build.fastllama import Model, ModelKind ModuleNotFoundError: No module named 'build.fastllama'
Closed a year ago8
n_ctx argument is ignored
Closed a year ago4
When stop words are reached, they get ingested, but are not forwarded to streaming_fn.
Closed a year ago4
convert-pth-to-ggml.py expects 2 parts for ALPACA-LORA-13B, but it has only one
Closed a year ago5
Cannot build this
Closed a year ago5
Passing arguments such as -ins
Closed a year ago2
RuntimeError: Unable to load model because of bad magic
Closed a year ago12
AVX2 performance issue
Closed a year ago17
Bad Magic error
Closed a year ago6
Lora adaptor support
Closed a year ago1
Cmake Error
Closed a year ago1
Is Alpaca 13B and 30B tested?
Closed 2 years ago5
Unicode characters break tokenizer
Closed a year ago18
ModuleNotFoundError: No module named 'fastLlama' after setup.py update
Closed a year ago14
function wrap for getting the embedding
Closed a year ago3
Add posibility to choose python version for module or make it independent from version
Closed a year ago11
Fix multiple relative pointer transform
Closed a year ago1
Return Log Probs in Output
Closed a year ago2
Problems while Trying to Run code programatically
Closed a year ago9
Error when using setup.py
Closed a year ago5
Make prompt ingestion faster!
Closed a year ago2
Still slow on AVX2 CPUs
Closed a year ago1
Does not support Python 3.11
Closed 2 years ago1
Error at ./build.sh
Closed 2 years ago8
Error using build.sh
Closed 2 years ago1
Getting Error with make command
Closed 2 years ago22
Example doc has incorrect repo
Closed 2 years ago1
Stop words is buggy!
Closed 2 years ago1
quantize.py is not build. quantize binary is.
Closed 2 years ago2
Unable to build bridge.cpp and link the 'libllama'
Closed 2 years ago2