ColdFusion2001

RAG_Techniques

This repository showcases various advanced techniques for Retrieval-Augmented Generation (RAG) systems. RAG systems combine information retrieval with generative models to provide accurate and contextually rich responses.

awesome-kan

A comprehensive collection of KAN(Kolmogorov-Arnold Network)-related resources, including libraries, projects, tutorials, papers, and more, for researchers and developers in the Kolmogorov-Arnold Network field.

doremi

Pytorch implementation of DoReMi, a method for optimizing the data mixture weights in language modeling datasets

RetrievalTutorials

gpt-prompt-engineer

ZeroEval

A simple unified framework for evaluating LLMs

MoDS

Intra-Fusion

Towards Meta-Pruning via Optimal Transport, ICLR 2024 (Spotlight)

meta-weight-net

NeurIPS'19: Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting (Pytorch implementation for noisy labels).

distilabel

Distilabel is a framework for synthetic data and AI feedback for engineers who need fast, reliable and scalable pipelines based on verified research papers.

matmulfreellm

Implementation for MatMul-free LM.

llmtools

Finetuning Large Language Models on One Consumer GPU in Under 4 Bits

complexity-scaling

gzip Predicts Data-dependent Scaling Laws

FlashRAG

⚡FlashRAG: A Python Toolkit for Efficient RAG Research

modula

Scalable neural net training via automatic normalization in the modular norm.

logix

AI Logging for Interpretability and Explainability🔬

LOMO

LOMO: LOw-Memory Optimization

OpenRLHF

An Easy-to-use, Scalable and High-performance RLHF Framework (70B+ PPO Full Tuning & Iterative DPO & LoRA & Mixtral)

model-explorer

A modern model graph visualizer and debugger

zett

Code for Zero-Shot Tokenizer Transfer

arc-dsl

Domain Specific Language for the Abstraction and Reasoning Corpus

KAN-GPT-2

Training small GPT-2 style models using Kolmogorov-Arnold networks.

kan-gpt

The PyTorch implementation of Generative Pre-trained Transformers (GPTs) using Kolmogorov-Arnold Networks (KANs) for language modeling

matryoshka-representation-learning

PyTorch implementation for MRL

energy-transformer-torch

Official Implementation of Energy Transformer in PyTorch for Mask Image Reconstruction

NeMo-Aligner

Scalable toolkit for efficient model alignment

pyreft

ReFT: Representation Finetuning for Language Models

pykan

Kolmogorov Arnold Networks

MiniMoE

Code for ACL 2023 paper titled "Lifting the Curse of Capacity Gap in Distilling Language Models"

llamaduo

This project showcases an LLMOps pipeline that fine-tunes a small-size LLM model to prepare for the outage of the service LLM. For this project, we have initially chosen Gemini 1.0 Pro for service type LLM and Gemma 2B/7B for small sized LLM model. It now supports other service LLMs such as GPT4 and Claude3.