ridgerchu

BAdam

SpectraSuite

triton

Development repository for the Triton language and compiler

matmulfreellm

Implementation for MatMul-free LM.

DoRA

[ICML2024 (Oral)] Official PyTorch implementation of DoRA: Weight-Decomposed Low-Rank Adaptation

LongLoRA

Code and documents of LongLoRA and LongAlpaca (ICLR 2024 Oral)

gpt-2

Code for the paper "Language Models are Unsupervised Multitask Learners"

ollama

Get up and running with Llama 3.1, Mistral, Gemma 2, and other large language models.

finetune-fuyu

example showing finetuning of fuyu

SAD

End-to-End Autonomous Driving with Spiking Neural Networks

BitBLAS

BitBLAS is a library to support mixed-precision matrix multiplications, especially for quantized LLM deployment.

efficient-kan

An efficient pure-PyTorch implementation of Kolmogorov-Arnold Network (KAN).

LinearAttentionArena

Here we will test various linear attention designs.

MedicalGPT

MedicalGPT: Training Your Own Medical GPT Model with ChatGPT Training Pipeline. 训练医疗大模型，实现了包括增量预训练(PT)、有监督微调(SFT)、RLHF、DPO、ORPO。

PFA

Official repository for paper: Tensor Decomposition Based Attention Module for Spiking Neural Networks

llm.c

LLM training in simple, raw C/CUDA

flash-linear-attention

Efficient implementations of state-of-the-art linear attention models in Pytorch and Triton

MagicTime

MagicTime: Time-lapse Video Generation Models as Metamorphic Simulators

rwkv

RWKV model implementation

HGRN

[NeurIPS 2023 spotlight] Official implementation of HGRN in our NeurIPS 2023 paper - Hierarchically Gated Recurrent Neural Network for Sequence Modeling

CANConv

Content-Adaptive Non-Local Convolution for Remote Sensing Pansharpening

hgru-pytorch

SFOD

This is the official implementation of the 'SFOD: Spiking Fusion Object Detection'.

Vision-RWKV

Vision-RWKV: Efficient and Scalable Visual Perception with RWKV-Like Architectures

ripunzip

MS-ResNet

Advancing Spiking Neural Networks towards Deep Residual Learning

ST-P3

[ECCV 2022] ST-P3, an end-to-end vision-based autonomous driving framework via spatial-temporal feature learning.

bullet

bullet: A Zero-Shot / Few-Shot Learning, LLM Based, text classification framework

mamba-minimal

Simple, minimal implementation of the Mamba SSM in one file of PyTorch.

RWKV-infctx-trainer

RWKV infctx trainer, for training arbitary context sizes, to 10k and beyond!