fp4

There are 2 repositories under fp4 topic.

NVIDIA / TransformerEngine
A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit and 4-bit floating point (FP8 and FP4) precision on Hopper, Ada and Blackwell GPUs, to provide better performance with lower memory utilization in both training and inference.
cuda deep-learning gpu machine-learning python pytorch fp8 jax fp4
Language:Python 2903
intel / neural-compressor
SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime
low-precision pruning sparsity auto-tuning knowledge-distillation quantization quantization-aware-training post-training-quantization smoothquant large-language-models awq fp4 gptq int4 int8 mxformat sparsegpt
Language:Python 2520
intel / neural-speed
An innovative library for efficient LLM inference via low-bit quantization
cpu fp8 gaudi2 gpu int4 int8 llm-inference low-bit sparsity fp4 llm-fine-tuning mxformat nf4 llamacpp int2 int3 int5 int6 int7 int1
Language:C++ 349
Tencent / AngelSlim
Model compression toolkit engineered for enhanced usability, comprehensiveness, and efficiency.
llm llm-compression quantization speculative-decoding diffusion vlm hunyuan deepseek qwen fp4
Language:Python 200
MurrellGroup / Microfloats.jl
Narrow precision floating point types
floating-point microfloat minifloat fp4 fp6 microscaling fp8
Language:Julia 10
mukullokhande99 / XR-NPE
Python implementations for multi-precision quantization in computer vision and sensor fusion workloads, targeting the XR-NPE Mixed-Precision SIMD Neural Processing Engine. The code includes visual inertial odometry (VIO), object classification, and eye gaze extraction code in FP4, FP8, Posit4, Posit8, and BF16 formats.
bf16 eye-gaze-detection fp4 fp8 object-detection posit quantization visual-inertial-odometry
Language:Jupyter Notebook 2
gm-stuffs / fairphone_fp4_dump
fairphone fp4 lito
Language:Shell

fp4

NVIDIA / TransformerEngine

intel / neural-compressor

intel / neural-speed

Tencent / AngelSlim

MurrellGroup / Microfloats.jl

mukullokhande99 / XR-NPE

gm-stuffs / fairphone_fp4_dump