IMNearth

Open-Sora

Open-Sora: Democratizing Efficient Video Production for All

MiniCPM-V

MiniCPM-V 2.6: A GPT-4V Level MLLM for Single Image, Multi Image and Video on Your Phone

UFO

A UI-Focused Agent for Windows OS Interaction.

MiniCPM

MiniCPM3-4B: An edge-side LLM that surpasses GPT-3.5-Turbo.

taming-transformers

Taming Transformers for High-Resolution Image Synthesis

latent-consistency-model

Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference

MGM

Official repo for "Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models"

MobileAgent

Mobile-Agent: The Powerful Mobile Device Operation Assistant Family

Qwen2-VL

Qwen2-VL is the multimodal large language model series developed by Qwen team, Alibaba Cloud.

nerfies.github.io

ReAct

[ICLR 2023] ReAct: Synergizing Reasoning and Acting in Language Models

Emu

Emu Series: Generative Multimodal Models from BAAI

mPLUG-DocOwl

mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understanding

Qwen-Audio

The official repo of Qwen-Audio (通义千问-Audio) chat & pretrained large audio language model proposed by Alibaba Cloud.

clash-for-linux

clash-for-linux

SwissArmyTransformer

SwissArmyTransformer is a flexible and powerful library to develop your own Transformer variants.

Show-o

Repository for Show-o, One Single Transformer to Unify Multimodal Understanding and Generation.

MiniGPT-5

Official implementation of paper "MiniGPT-5: Interleaved Vision-and-Language Generation via Generative Vokens"

groundingLMM

[CVPR 2024 🔥] Grounding Large Multimodal Model (GLaMM), the first-of-its-kind model capable of generating natural language responses that are seamlessly integrated with object segmentation masks.

self-refine

LLMs can generate feedback on their work, use it to improve the output, and repeat this process iteratively.

InstructDiffusion

PyTorch implementation of InstructDiffusion, a unifying and generic framework for aligning computer vision tasks with human instructions.

InfLLM

The code of our paper "InfLLM: Unveiling the Intrinsic Capacity of LLMs for Understanding Extremely Long Sequences with Training-Free Memory"

SeeClick

The model, data and code for the visual GUI Agent SeeClick

android_world

AndroidWorld is an environment and benchmark for autonomous agents

webui

screen_qa

ScreenQA dataset was introduced in the "ScreenQA: Large-Scale Question-Answer Pairs over Mobile App Screenshots" paper. It contains ~86K question-answer pairs collected by human annotators for ~35K screenshots from Rico. It should be used to train and evaluate models capable of screen content understanding via question answering.

GUICourse

GUICourse: From General Vision Langauge Models to Versatile GUI Agents

TextHawk

Exploring Efficient Fine-Grained Perception of Multimodal Large Language Models

CoAT

Official implementation for "Android in the Zoo: Chain-of-Action-Thought for GUI Agents" (Findings of EMNLP 2024)

VoCoT

VoCoT: Unleashing Visually Grounded Multi-Step Reasoning in Large Multi-Modal Models