davidchern

MemoryMosaics

Memory Mosaics are networks of associative memories working in concert to achieve a prediction task.

LARS-VSA

MoRA

MoRA: High-Rank Updating for Parameter-Efficient Fine-Tuning

Hrrformer

Hrrformer: A Neuro-symbolic Self-attention Model (ICML23)

DCFormer

Convolutional-KANs

This project extends the idea of the innovative architecture of Kolmogorov-Arnold Networks (KAN) to the Convolutional Layers, changing the classic linear transformation of the convolution to learnable non linear activations in each pixel.

AGI-survey

nanoXLSTM

The simplest, fastest repository for training/finetuning medium-sized xLSTMs.

MAP-NEO

fast-kan

FastKAN: Very Fast Implementation of Kolmogorov-Arnold Networks (KAN)

LKAN

Variations of Kolmogorov-Arnold Networks

ferns

Fast Exact Retrieval for Nearest-neighbor Search

ChebyKAN

Kolmogorov-Arnold Networks (KAN) using Chebyshev polynomials instead of B-splines.

FCN-KAN

Kolmogorov–Arnold Networks with modified activation (using fully connected network to represent the activation)

drl-zh

Deep Reinforcement Learning: Zero to Hero!

kanrl

Kolmogorov-Arnold Network for Reinforcement Leaning, initial experiments

efficient-kan

An efficient pure-PyTorch implementation of Kolmogorov-Arnold Network (KAN).

FourierKAN

NCHL

Neuron centric Hebbian Learning

pykan

Kolmogorov Arnold Networks

MNBVC

MNBVC(Massive Never-ending BT Vast Chinese corpus)超大规模中文语料集。对标chatGPT训练的40T数据。MNBVC数据集不但包括主流文化，也包括各个小众文化甚至火星文的数据。MNBVC数据集包括新闻、作文、小说、书籍、杂志、论文、台词、帖子、wiki、古诗、歌词、商品介绍、笑话、糗事、聊天记录等一切形式的纯文本中文数据。

Token-level-Direct-Preference-Optimization

Reference implementation for Token-level Direct Preference Optimization(TDPO)

data-juicer

A one-stop data processing system to make data higher-quality, juicier, and more digestible for LLMs! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷为大语言模型提供更高质量、更丰富、更易”消化“的数据！

Score-Entropy-Discrete-Diffusion

[ICML 2024 Oral] Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution (https://arxiv.org/abs/2310.16834)

infini-mini-transformer

This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and training code.

YuLan-Chat

YuLan-Chat: An Open-Source Bilingual Chatbot

LLMBox

A comprehensive library for implementing LLMs, including a unified training pipeline and comprehensive model evaluation.

Bloom-Lora

Finetune Bloom big language model with Lora method

ColossalAI

Making large AI models cheaper, faster and more accessible

Steel-LLM

Train a Chinese LLM From 0 by Personal