lyogavin / Anima

33B Chinese LLM, DPO QLORA, 100K context, AirLLM 70B inference with single 4GB GPU

lyogavin/Anima Issues

Insuficient disk space
Updated 17 days ago2
mac m2 run air llm garage-bAInd/Platypus2-7B get error Input must be a file-like object opened in binary mode, or string
Updated 19 days ago3
segmentation fault python3 airllm2.py
Updated 20 days ago3
safetensors_rust.SafetensorError: Error while deserializing header: MetadataIncompleteBuffer
Updated 20 days ago
error in apple mac m3
Updated 20 days ago1
CPU ram offload
Updated 20 days ago
Discord Invite Expired in the readme
Updated 21 days ago1
Does airllm support quantized gguf/gptq/awq models ?
Updated 21 days ago
COMPILED_WITH_CUDA error requires libcuda.so
Updated 22 days ago
Error with Llama3: ValueError: Trying to set a tensor of shape torch.Size([1024, 8192]) in "weight" (which has shape torch.Size([8192, 8192])), this look incorrect.
Updated 23 days ago
AirLLM: Support for DirectML
Updated 23 days ago1
跑不通chatglm3，请大佬指教。
Updated 24 days ago2
to run llama3-70b,but fail to import. why?
Closed 24 days ago
Generation takes forever
Updated 25 days ago2
Any CoreML implementation plans?
Updated 25 days ago
Mac 'str' object has no attribute 'sequences
Updated 25 days ago
Running on Mac get traceback error
Closed 25 days ago1
"src" directory name is conflicted
Updated a month ago
how to delete the original download model after it has been downloaded
Updated a month ago
通过Ollama下载了的模型，如何在airllm中直接使用呢
Updated a month ago1
Is it possible to use AirLLM with a quantized input model?
Updated a month ago3
请求支持llama3
Closed a month ago2
The following error is encountered when running the sample code
Updated 2 months ago
compression parameter on mac.dosent work.
Updated 2 months ago
Macbook "Torch not compiled with CUDA enabled" Error
Closed 4 months ago2
Support for OPT Architecture
Updated 2 months ago
For me this model is extremely underperforming
Updated 2 months ago1
attn impl to sdpa...
Updated 3 months ago3
似乎只能产生很少的字符
Closed 3 months ago2
Add UI like AUTOMATIC1111 for stable-diffusion-webui
Updated 3 months ago
Which 70B model does macOS support?
Updated 3 months ago
ValueError: LlamaForCausalLM does not support an attention implementation through torch.nn.functional.scaled_dot_product_attention yet.
Updated 4 months ago1
Optimize for consumer GPU, eg 11GB or 16GB
Updated 4 months ago
AMD gpu support
Updated 4 months ago
用airllm运行Yi-34B-chat模型，分层之后报这个错误
Updated 4 months ago1
Will the airllm framework be adapted for the streaming output functionality of different models in the future?
Updated 4 months ago
关于对话模型是否能使用airllm
Updated 4 months ago1
AirLLMLlamaMlx fails to load model with mlx==0.0.7
Updated 4 months ago
how to infer on multiple gpus?
Closed 4 months ago1
Finetune 70B on 24GB 4090?
Updated 4 months ago1
ValueError: max() arg is an empty sequence(Apple M2 Max, macOS 14.2.1)
Updated 4 months ago6
safetensors_rust.SafetensorError: Error while deserializing header: HeaderTooLarge
Updated 5 months ago1
microsoft-phi2:max() arg is an empty sequence
Updated 5 months ago1
ImportError: cannot import name AutoMode
Closed 5 months ago1
Would adding Parallelism speed up AirLLM?
Updated 5 months ago
Mac quantization
Updated 5 months ago
Mac Airllm Inference tigerbot-70b-chat-v2
Updated 5 months ago
configure the chunk split size
Updated 5 months ago
Mixtral models seem to run forever
Closed 5 months ago1
Does Airllm support sqlcoder-34b which was fine-tuned on codellama?
Closed 5 months ago1