yihaocs

pyautogui

A cross-platform GUI automation Python module for human beings. Used to programmatically control the mouse & keyboard.

self-operating-computer

A framework to enable multimodal models to operate a computer.

DeepSeek-Coder

DeepSeek Coder: Let the Code Write Itself

openchat

OpenChat: Advancing Open-source Language Models with Imperfect Data

VideoCrafter

VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models

latent-consistency-model

Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference

vimGPT

Browse the web with GPT-4V and Vimium

PixArt-alpha

PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis

consistencydecoder

Consistency Distilled Diff VAE

i2vgen-xl

Official repo for VGen: a holistic video generation ecosystem for video generation building on diffusion models

self-rag

This includes the original implementation of SELF-RAG: Learning to Retrieve, Generate and Critique through self-reflection by Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi.

tarsier

Vision utilities for web interaction agents 👀

GPT-4V-Act

AI agent using GPT-4V(ision) capable of using a mouse/keyboard to interact with web UI

ReWOO

Decoupling Reasoning from Observations for Efficient Augmented Language Models

SALMONN

SALMONN: Speech Audio Language Music Open Neural Network

MFTCoder

High Accuracy and efficiency multi-task fine-tuning framework for Code LLMs. This work has been accepted by KDD 2024.

EvaluationPapers4ChatGPT

Resource, Evaluation and Detection Papers for ChatGPT

lumos

Code and data for "Lumos: Learning Agents with Unified Data, Modular Design, and Open-Source LLMs"

Youku-mPLUG

Youku-mPLUG: A 10 Million Large-scale Chinese Video-Language Pre-training Dataset and Benchmarks

llm-decontaminator

Code for the paper "Rethinking Benchmark and Contamination for Language Models with Rephrased Samples"

GPT-V-on-Web

👀🧠 GPT-4 Vision x 💪⌨️ Vimium = Autonomous Web Agent

cpl

Code for Contrastive Preference Learning (CPL)

VideoLDM

Unofficial PyTorch implementation of the VideoLDM.

LVIS-INSTRUCT4V

generative-elicitation

MM-Navigator

LMMs as Smartphone Agents

DuckTrack

Multimodal computer agent data collection program

ScaleLong

The official repository of paper "ScaleLong: Towards More Stable Training of Diffusion Model via Scaling Network Long Skip Connection" (NeurIPS 2023)

VLC

Research code for "Training Vision-Language Transformers from Captions Alone"

Interactive-RAG