KelleyYin

llama_index

LlamaIndex is a data framework for your LLM applications

GPT-SoVITS

1 min voice data can also be used to train a good TTS model! (few shot voice cloning)

LLaMA-Factory

Efficiently Fine-Tune 100+ LLMs in WebUI (ACL 2024)

insightface

State-of-the-art 2D and 3D Face Analysis Project

analysis-ik

🚌 The IK Analysis plugin integrates Lucene IK analyzer into Elasticsearch and OpenSearch, support customized dictionary.

tiktoken

tiktoken is a fast BPE tokeniser for use with OpenAI's models.

mistral-inference

Official inference library for Mistral models

accelerate

🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support

AutoGPTQ

An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.

NeMo-Guardrails

NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.

wikiextractor

A tool for extracting plain text from Wikipedia dumps

work-in-australia

Work in Australia as a Developer / 程序员如何申请到澳洲工作

GenerativeAIExamples

Generative AI reference workflows optimized for accelerated infrastructure and microservice architecture.

direct-preference-optimization

Reference implementation for DPO (Direct Preference Optimization)

DeepSpeed-MII

MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.

FinGLM

FinGLM: 致力于构建一个开放的、公益的、持久的金融大模型项目，利用开源开放来促进「AI+金融」。

pyserini

Pyserini is a Python toolkit for reproducible information retrieval research with sparse and dense representations.

yarn

YaRN: Efficient Context Window Extension of Large Language Models

bigscience

Central place for the engineering/scaling WG: documentation, SLURM scripts and logs, compute environment and data.

HuggingFace-Download-Accelerator

利用HuggingFace的官方下载工具从镜像网站进行高速下载。

papermage

library supporting NLP and CV research on scientific papers

Chinese-Mixtral-8x7B

中文Mixtral-8x7B（Chinese-Mixtral-8x7B）

Pai-Megatron-Patch

The official repo of Pai-Megatron-Patch for LLM & VLM large scale training developed by Alibaba Cloud.

Wandb_Tutorial

How to use wandb?

DISC-LawLLM

DISC-LawLLM, an intelligent legal system utilizing large language models (LLMs) to provide a wide range of legal services

LEval

[ACL'24 Outstanding] Data and code for L-Eval, a comprehensive long context language models evaluation benchmark

hf_transfer

pdf_paragraphs_extraction

vllm-client

vLLM client with minimal dependencies

Semantic-Search