davidchern

ColossalAI

Making large AI models cheaper, faster and more accessible

pykan

Kolmogorov Arnold Networks

efficient-kan

An efficient pure-PyTorch implementation of Kolmogorov-Arnold Network (KAN).

MNBVC

MNBVC(Massive Never-ending BT Vast Chinese corpus)超大规模中文语料集。对标chatGPT训练的40T数据。MNBVC数据集不但包括主流文化，也包括各个小众文化甚至火星文的数据。MNBVC数据集包括新闻、作文、小说、书籍、杂志、论文、台词、帖子、wiki、古诗、歌词、商品介绍、笑话、糗事、聊天记录等一切形式的纯文本中文数据。

data-juicer

A one-stop data processing system to make data higher-quality, juicier, and more digestible for (multimodal) LLMs! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷为大模型提供更高质量、更丰富、更易”消化“的数据！

drl-zh

Deep Reinforcement Learning: Zero to Hero!

MAP-NEO

Convolutional-KANs

This project extends the idea of the innovative architecture of Kolmogorov-Arnold Networks (KAN) to the Convolutional Layers, changing the classic linear transformation of the convolution to learnable non linear activations in each pixel.

FourierKAN

LLMBox

A comprehensive library for implementing LLMs, including a unified training pipeline and comprehensive model evaluation.

YuLan-Chat

YuLan: An Open-Source Large Language Model

Score-Entropy-Discrete-Diffusion

[ICML 2024 Best Paper] Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution (https://arxiv.org/abs/2310.16834)

ChebyKAN

Kolmogorov-Arnold Networks (KAN) using Chebyshev polynomials instead of B-splines.

MoRA

MoRA: High-Rank Updating for Parameter-Efficient Fine-Tuning

fast-kan

FastKAN: Very Fast Implementation of Kolmogorov-Arnold Networks (KAN)

AGI-survey

kanrl

Kolmogorov-Arnold Network for Reinforcement Leaning, initial experiments

DCFormer

Steel-LLM

Train a Chinese LLM From 0 by Personal

LKAN

Variations of Kolmogorov-Arnold Networks

FCN-KAN

Kolmogorov–Arnold Networks with modified activation (using fully connected network to represent the activation)

Token-level-Direct-Preference-Optimization

Reference implementation for Token-level Direct Preference Optimization(TDPO)

infini-mini-transformer

This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and training code.

Hrrformer

Hrrformer: A Neuro-symbolic Self-attention Model (ICML23)

nanoXLSTM

The simplest, fastest repository for training/finetuning medium-sized xLSTMs.

MemoryMosaics

Memory Mosaics are networks of associative memories working in concert to achieve a prediction task.

Bloom-Lora

Finetune Bloom big language model with Lora method

ferns

Fast Exact Retrieval for Nearest-neighbor Search

NCHL

Neuron centric Hebbian Learning

LARS-VSA