justHungryMan

ollama

Get up and running with Llama 3.2, Mistral, Gemma 2, and other large language models.

llama.cpp

LLM inference in C/C++

llama

Inference code for Llama models

llama-recipes

Scripts for fine-tuning Meta Llama with composable FSDP & PEFT methods to cover single/multi-node GPUs. Supports default & custom datasets for applications such as summarization and Q&A. Supporting a number of candid inference solutions such as HF TGI, VLLM for local or cloud deployment. Demo apps to showcase Meta Llama for WhatsApp & Messenger.

ZLUDA

CUDA on non-NVIDIA GPUs

OLMo

Modeling, training, eval, and inference code for OLMo

whisper-vits-svc

Core Engine of Singing Voice Conversion & Singing Voice Clone

diff-svc

Singing Voice Conversion via diffusion model

LLMDataHub

A quick guide (especially) for trending instruction finetuning datasets

EasyLM

Large language models (LLMs) made easy, EasyLM is a one stop solution for pre-training, finetuning, evaluating and serving LLMs in JAX/Flax.

datatrove

Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.

magicoder

[ICML'24] Magicoder: Empowering Code Generation with OSS-Instruct

LLMTest_NeedleInAHaystack

Doing simple retrieval from LLM models at various context lengths to measure accuracy

resource-stream

CUDA related news and material links

zsh-in-docker

Install Zsh, Oh My Zsh and plugins inside a Docker container with one line!

datacomp

DataComp: In search of the next generation of multimodal datasets

KICE_slayer_AI_Korean

수능 국어 1등급에 도전하는 AI

newspaper4k

📰 Newspaper4k a fork of the beloved Newspaper3k. Extraction of articles, titles, and metadata from news websites.

text-clustering

Easily embed, cluster and semantically label text datasets

cosmopedia

honeybee

Official implementation of project Honeybee (CVPR 2024)

doremi

Pytorch implementation of DoReMi, a method for optimizing the data mixture weights in language modeling datasets

OBELICS

Code used for the creation of OBELICS, an open, massive and curated collection of interleaved image-text web documents, containing 141M documents, 115B text tokens and 353M images.

xlora

X-LoRA: Mixture of LoRA Experts

42dot_LLM

42dot LLM consists of a pre-trained language model, 42dot LLM-PLM, and a fine-tuned model, 42dot LLM-SFT, which is trained to respond to user prompts and supports both languages simultaneously by training a large amount of Korean and English text.

season2

Jiphyeonjeon Season 2

PseudoChat-Instruction-Dataset

mmlu

Measuring Massive Multitask Language Understanding | ICLR 2021

wikipedia-markdown-generator

A simple python script to convert any Wikipedia article to Markdown.

text-anonymization

A guide to anonymize text effortlessly using Presidio, an open-source library developed by Microsoft.