Spycsh

prometheus-fastapi-instrumentator

Instrument your FastAPI with Prometheus metrics.

GenAIEval

Evaluation, benchmark, and scorecard, targeting for performance on throughput and latency, accuracy on popular evaluation harness, safety, and hallucination

jaeger

CNCF Jaeger, a Distributed Tracing Platform

GenAIComps

GenAI components at micro-service level; GenAI service composer to create mega-service

docarray

Represent, send, store and search multimodal data

llama3

The official Meta Llama 3 GitHub site

GPT-SoVITS

1 min voice data can also be used to train a good TTS model! (few shot voice cloning)

gemma_pytorch

The official PyTorch implementation of Google's Gemma models

gpt-fast

Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.

LoRA

Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"

kserve

Standardized Serverless ML Inference Platform on Kubernetes

TensorRT-LLM

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.

FasterTransformer

Transformer related optimization, including BERT, GPT

server

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

noisereduce

Noise reduction in python using spectral gating (speech, bioacoustics, audio, time-domain signals)

NeMo

A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)

text-to-text-transfer-transformer

Code for the paper "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer"

tensorflow-onnx

Convert TensorFlow, Keras, Tensorflow.js and Tflite models to ONNX

notebooks

Jupyter notebooks for the Natural Language Processing with Transformers book

intel-extension-for-transformers

⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Platforms⚡

oneDNN

oneAPI Deep Neural Network Library (oneDNN)

optimization-manual

Contains the source code examples described in the "Intel® 64 and IA-32 Architectures Optimization Reference Manual"

Awesome-Pruning

A curated list of neural network pruning resources.

awesome-ml-model-compression

Awesome machine learning model compression research papers, quantization, tools, and learning material.

MetaNN

oneAPI_course

oneAPI - Data Parallel C++ course for students

streamingbook

Code snippets from the Streaming Systems book (streamingbook.net).

manim

Animation engine for explanatory math videos

neural-compressor

SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime

statefun-aws-lambda-demo