InftyAI

llmaz

☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!

Awesome-LLMOps

🎉 An awesome & curated list of best LLMOps tools.

Manta

💫 A lightweight p2p-based cache system for model distributions on Kubernetes. Reframing now to make it an unified cache system with POSIX promise 🎯

AMRS

The adaptive model routing system for exploration and exploitation.

scheduler-plugins

Scheduler-Plugins maintains a list of kubernetes scheduler plugins used in InftyAI community.

alphatrion

AlphaTrion is an open-source framework to help build GenAI applications, including experiment tracking, adaptive model routing, prompt optimization and performance evaluation.

PR-Copilot

Your AI pair programmer 🤖️ specialized in code review, code summary and even code completion. 🧑‍💻🐛

omnistore

🎯 An unified python client to communicate with various kinds of object-store providers.

website

Website and Blogs.

.github

community

The InftyAI community.

karpenter

This is a fork of Karpenter focused on GPU scaling with llmaz.

template-repo

A template repo.