Wang-Xiaodong1899

pytubefix

A pytube fork with additional features and fixes

Open-LLaVA-NeXT

An open-source implementation of LLaVA-NeXT.

video2dataset

Easily create large video dataset from video urls

HERO

Research code for EMNLP 2020 paper "HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training"

NExT-QA

NExT-QA: Next Phase of Question-Answering to Explaining Temporal Actions (CVPR'21)

POPE

[EMNLP'23] The official GitHub page for ''Evaluating Object Hallucination in Large Vision-Language Models''

ScienceQA

Data and code for NeurIPS 2022 Paper "Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering".

LLMs-from-scratch

Implementing a ChatGPT-like LLM in PyTorch from scratch, step by step

fish-speech

Brand new TTS solution

lmms-eval

Accelerating the development of large multimodal models (LMMs) with lmms-eval

MM-Instruct

MM-Instruct: Generated Visual Instructions for Large Multimodal Model Alignment

Video-Infinity

Video-Infinity generates long videos quickly using multiple GPUs without extra training.

Firefly

Firefly: 大模型训练工具，支持训练Qwen2、Yi1.5、Phi-3、Llama3、Gemma、MiniCPM、Yi、Deepseek、Orion、Xverse、Mixtral-8x7B、Zephyr、Mistral、Baichuan2、Llma2、Llama、Qwen、Baichuan、ChatGLM2、InternLM、Ziya2、Vicuna、Bloom等大模型

ChatTTS

A generative speech model for daily dialogue.

POVID

[Arxiv] Aligning Modalities in Vision Large Language Models via Preference Fine-tuning

mPLUG-HalOwl

mPLUG-HalOwl: Multimodal Hallucination Evaluation and Mitigating

bootstrapped-preference-optimization-BPO-

code for "Strengthening Multimodal Large Language Model with Bootstrapped Preference Optimization"

RLHF-V

[CVPR'24] RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback

SPIN

The official implementation of Self-Play Fine-Tuning (SPIN)

cambrian

Cambrian-1 is a family of multimodal LLMs with a vision-centric design.

LOOK-M

Official implementation of "LOOK-M: Look-Once Optimization in KV Cache for Efficient Multimodal Long-Context Inference"

LLaVA-Hound-DPO

SRT

i-SRT:Aligning Large Multimodal Models for Videos by Iterative Self-Retrospective Judgement

vlm-rlaif

ACL'24 Main track

elevenlabs-python

The official Python API for ElevenLabs Text to Speech.

Bark-Voice-Cloning

Bark Voice Cloning and Voice Cloning for Chinese Speech

SeVa

Official code of paper "Self-Supervised Visual Preference Alignment" https://arxiv.org/abs/2404.10501

vlm_arm

机械臂+大模型+多模态=人机协作具身智能体

Reinforcement-Learning-in-Robotics

This is a private learning repository for reinforcement learning techniques used in robotics.

multimodal-dit-pytorch

Implementation of a multimodal diffusion transformer in Pytorch