fp4

There are 1 repository under fp4 topic.

intel / neural-compressor
SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime
auto-tuning awq fp4 gptq int4 int8 knowledge-distillation large-language-models low-precision mxformat post-training-quantization pruning quantization quantization-aware-training smoothquant sparsegpt sparsity
Language:Python 2074
intel / neural-speed
An innovative library for efficient LLM inference via low-bit quantization
cpu fp8 gaudi2 gpu int4 int8 llm-inference low-bit sparsity fp4 llm-fine-tuning mxformat nf4 llamacpp int2 int3 int5 int6 int7 int1
Language:C++ 314
gm-stuffs / fairphone_fp4_dump
fairphone fp4 lito
Language:Shell