Yard1

Ray-DeepSpeed-Inference

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

Ray-StableDiffusion

pdx-steam-workshop-publisher-action

Automatically upload your Hearts of Iron IV (support for more Paradox games to come) mod to Steam Workshop.

HoI4-GFX-Search

This tool helps you easily find GFX from unmodded (vanilla) Hearts of Iron 4 1.14.

hoi4_headless

A Docker container to upload Hearts of Iron IV mods automatically

lm-evaluation-harness

A framework for few-shot evaluation of language models.

text-generation-inference

Large Language Model Text Generation Inference

DeepSpeed

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

lm-format-enforcer

Enforce the output format (JSON Schema, Regex etc) of a language model

megablocks

ray

A fast and simple framework for building and running distributed applications. Ray is packaged with RLlib, a scalable reinforcement learning library, and Tune, a scalable hyperparameter tuning library.

Ray-LLM-script

ray-serve-deepspeed

Run deepspeed on ray serve

accelerate

🚀 A simple way to train and use PyTorch models with multi-GPU, TPU, mixed-precision

cupy

NumPy & SciPy for GPU

flashinfer

FlashInfer: Kernel Library for LLM Serving

gpt-neox

An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library.

gradio

Create UIs for your machine learning model in Python in 3 minutes

joblib

Computing with Python functions.

lightgbm_ray

LightGBM on Ray

llama-cpp-python

Python bindings for llama.cpp

LLM-Ray

pyod

A Comprehensive and Scalable Python Library for Outlier Detection (Anomaly Detection)

RWKV-LM

RWKV is a RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.

trlx

A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)

vllm-flash-attention

Fast and memory-efficient exact attention

Whisper-Ray

xgboost_ray

Distributed XGBoost on Ray

Yard1