int4

There are 0 repository under int4 topic.

intel / neural-compressor
SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime
auto-tuning awq fp4 gptq int4 int8 knowledge-distillation large-language-models low-precision mxformat post-training-quantization pruning quantization quantization-aware-training smoothquant sparsegpt sparsity
Language:Python 2520
intel / auto-round
Advanced Quantization Algorithm for LLMs and VLMs, with support for CPU, Intel GPU, CUDA and HPU.
int4 mxfp4 nvfp4 quantization rounding transformers vllm
Language:Python 697
tpoisonooo / how-to-optimize-gemm
row-major matmul optimization
gemm-optimization armv7 arm64 cuda cuda-kernel ptx vulkan int4
Language:C++ 684
intel / neural-speed
An innovative library for efficient LLM inference via low-bit quantization
cpu fp8 gaudi2 gpu int4 int8 llm-inference low-bit sparsity fp4 llm-fine-tuning mxformat nf4 llamacpp int2 int3 int5 int6 int7 int1
Language:C++ 349
Danaozhong / rust-bitwriter
rust library to write integer types of any bit length into a buffer - from `i1` to `i64`.
bitbuffer bytebuffer int4 rust int15 bitbanging bitwriter
Language:Rust 3
ambv231 / tinyllama-coreml-ios18-quantization
Quantize TinyLlama-1.1B-Chat from PyTorch to CoreML (float16, int8, int4) for efficient on-device inference on iOS 18+.
ai apple-silicon coreml huggingface int4 int8 ios18 llama llm mlpackage mobile nlp on-device pytorch quantization tinyllama transformers
Language:Python 1
GreenBull31 / tinyllama-coreml-ios18-quantization
Quantize TinyLlama-1.1B-Chat from PyTorch to CoreML (float16, int8, int4) for efficient on-device inference on iOS 18+.
ai apple-silicon coreml huggingface int4 int8 ios18 llama llm mlpackage mobile nlp on-device pytorch quantization tinyllama transformers
Language:Python

int4

intel / neural-compressor

intel / auto-round

tpoisonooo / how-to-optimize-gemm

intel / neural-speed

Danaozhong / rust-bitwriter

ambv231 / tinyllama-coreml-ios18-quantization

GreenBull31 / tinyllama-coreml-ios18-quantization