ZCMax

GPT-SoVITS

1 min voice data can also be used to train a good TTS model! (few shot voice cloning)

CLIP

CLIP (Contrastive Language-Image Pretraining), Predict the most relevant text snippet given an image

llama3

The official Meta Llama 3 GitHub site

Open-Sora

Open-Sora: Democratizing Efficient Video Production for All

ml-ferret

rerun

Visualize streams of multimodal data. Fast, easy to use, and simple to integrate. Built in Rust using egui.

MGM

Official repo for "Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models"

Awesome-LLMs-for-Video-Understanding

🔥🔥🔥Latest Papers, Codes and Datasets on Vid-LLMs.

Awesome-LLM-3D

Awesome-LLM-3D: a curated list of Multi-modal Large Language Model in 3D world Resources

LLaVA-pp

🔥🔥 LLaVA++: Extending LLaVA with Phi-3 and LLaMA-3 (LLaVA LLaMA-3, LLaVA Phi-3)

Chat-UniVi

[CVPR 2024 Highlight🔥] Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understanding

LLaVA-Plus-Codebase

LLaVA-Plus: Large Language and Vision Assistants that Plug and Learn to Use Skills

VLMEvalKit

Open-source evaluation toolkit of large vision-language models (LVLMs), support GPT-4v, Gemini, QwenVLPlus, 40+ HF models, 20+ benchmarks

Mask3D

Mask3D predicts accurate 3D semantic instances achieving state-of-the-art on ScanNet, ScanNet200, S3DIS and STPLS3D.

vstar

PyTorch Implementation of "V* : Guided Visual Search as a Core Mechanism in Multimodal LLMs"

qingwu-zimu

青梧字幕是一款基于whisper的AI字幕提取工具

omnidata

A Scalable Pipeline for Making Steerable Multi-Task Mid-Level Vision Datasets from 3D Scans [ICCV 2021]

PLLaVA

Official repository for the paper PLLaVA

Stratified-Transformer

Stratified Transformer for 3D Point Cloud Segmentation (CVPR 2022)

Awesome-Open-AI-Sora

Sora AI Awesome List – Your go-to resource hub for all things Sora AI, OpenAI's groundbreaking model for crafting realistic scenes from text. Explore a curated collection of articles, videos, podcasts, and news about Sora's capabilities, advancements, and more.

probe3d

[CVPR 2024] Probing the 3D Awareness of Visual Foundation Models

NExT-Chat

The code of the paper "NExT-Chat: An LMM for Chat, Detection and Segmentation".

3D-VLA

Source codes for "3D-VLA: A 3D Vision-Language-Action Generative World Model"

open-eqa

OpenEQA Embodied Question Answering in the Era of Foundation Models

multi_token

Embed arbitrary modalities (images, audio, documents, etc) into large language models.

SceneVerse

VQASynth

Compose multimodal datasets 🎹

act3d-chained-diffuser

A unified architecture for multimodal multi-task robotic policy learning.

Online3D

[CVPR 2024] Memory-based Adapters for Online 3D Scene Perception

ScanRefer_Browser