EricLBuehler

mistral.rs

Blazingly fast LLM inference.

candle-vllm

Efficent platform for inference and serving local LLMs including an OpenAI compatible API server.

xlora

X-LoRA: Mixture of LoRA Experts

candle-lora

Low rank adaptation (LoRA) for Candle.

trc

A faster Arc.

candle-sampling

Sampling techniques for Candle.

candle-lora-macro

Macros for candle-lora.

candle_llm_dataset

A LLM-specific dataset for Candle.

candle

Minimalist ML framework for Rust

candle-cublaslt

flexrc

Fast, flexible, and safe reference counted type.

simd_matmul

O(n^2) matmul with SIMD.

dora

DORA (Dataflow-Oriented Robotic Application) is middleware designed to streamline and simplify the creation of AI-based robotic applications. It offers low latency, composable, and distributed dataflow capabilities. Applications are modeled as directed graphs, also referred to as pipelines.

merlin_rust

Make your code fly.

pyo3_smd_example

Example of pyo3_special_method_derive

range-checked

Range checked types leveraging Rust's type system.

candle-ext

An extension library to Candle that provides PyTorch functions not currently available in Candle

candle-layer-norm

peft

🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

EricLBuehler

candle-paged-attention

candle-rotary

cudarc

Safe rust wrapper around CUDA toolkit

docs

llama_index

LlamaIndex is a data framework for your LLM applications

mirage

A multi-level tensor algebra superoptimizer

ratchet

A cross-platform browser ML framework.

tch-rs

Rust bindings for the C++ api of PyTorch.

tokenizers

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs