m-pedro

DeepSeek-MoE

DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models

Look-into-MoEs

A Closer Look into Mixture-of-Experts in Large Language Models

Step-DPO

Implementation for "Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMs"

lbt

Can LLMs Learn by Teaching? A Preliminary Study

aimo-progress-prize

mergekit

Tools for merging pretrained large language models.

refusal_direction

Code and results accompanying the paper "Refusal in Language Models Is Mediated by a Single Direction".

DeepSeek-Coder-V2

DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence

dclm

DataComp for Language Models

HiddenMambaAttn

Official PyTorch Implementation of "The Hidden Attention of Mamba Models"

samplernn-pytorch

PyTorch implementation of SampleRNN: An Unconditional End-to-End Neural Audio Generation Model

flash-linear-attention

Efficient implementations of state-of-the-art linear attention models in Pytorch and Triton

Samba

Official implementation of "Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling"

mdlm

Simplified Masked Diffusion Language Model

scaling-with-vocab

📈 Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies

unsloth

Finetune Llama 3, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory

exa-crewai

magpie

Official repository for "Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing". Your efficient and high-quality synthetic data generation pipeline!

generative-ai-amazon-bedrock-langchain-agent-example

distributed-kge-poplar

The application is a end-user training and evaluation system for standard knowledge graph embedding models. It was developed to optimise the WikiKG90Mv2 dataset

graphcast

ThoughtSource

A central, open resource for data and tools related to chain-of-thought reasoning in large language models. Developed @ Samwald research group: https://samwald.info/

amazon-sagemaker-generativeai

Repository for training and deploying Generative AI models, including text-text, text-to-image generation and prompt engineering playground using SageMaker Studio.

guidance-for-a-multi-tenant-generative-ai-gateway-with-cost-and-usage-tracking-on-aws

This Guidance demonstrates how to build an internal Software-as-a-Service (SaaS) platform that provides access to foundation models, like those available through Amazon Bedrock, to different business units or teams within your organization

Lynx-hallucination-detection

RydbergGPT

Our LLM for Rydberg atom physics

zeroshot-classifier

Notebooks for training universal 0-shot classifiers on many different tasks

summer-school-transformers-2023

Course repository for the session "Hands-on Transformers: Fine-Tune your own BERT and GPT" of the Data Science Summer School 2023

grade-school-math

NeMo-Skills

A pipeline to improve skills of large language models