eric-xw

whisper

Robust Speech Recognition via Large-Scale Weak Supervision

segment-anything

The repository provides code for running inference with the SegmentAnything Model (SAM), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.

stablediffusion

High-Resolution Image Synthesis with Latent Diffusion Models

generative-models

Generative Models by Stability AI

Grounded-Segment-Anything

Grounded SAM: Marrying Grounding DINO with Segment Anything & Stable Diffusion & Recognize Anything - Automatically Detect , Segment and Generate Anything

gaussian-splatting

Original reference implementation of "3D Gaussian Splatting for Real-Time Radiance Field Rendering"

LAVIS

LAVIS - A One-stop Library for Language-Vision Intelligence

instruct-pix2pix

Voyager

An Open-Ended Embodied Agent with Large Language Models

OpenAGI

OpenAGI: When LLM Meets Domain Experts

LVM

MetaTransformer

Meta-Transformer for Unified Multimodal Learning

Multimodal-GPT

Multimodal-GPT

Transformer-in-Vision

Recent Transformer-based CV and related works.

Neural-Network-Diffusion

We introduce a novel approach for parameter generation, named neural network parameter diffusion (p-diff), which employs a standard latent diffusion model to synthesize a new set of parameters

AWSIM

Open source simulator for self-driving vehicles

Multi-Modality-Arena

Chatbot Arena meets multi-modality! Multi-Modality Arena allows you to benchmark vision-language models side-by-side while providing images as inputs. Supports MiniGPT-4, LLaMA-Adapter V2, LLaVA, BLIP-2, and many more!

Structured-Diffusion-Guidance

Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesis

MatrixCity

habitat-matterport3d-dataset

This repository contains code to reproduce experimental results from our HM3D paper in NeurIPS 2021.

LLMScore

LLMScore: Unveiling the Power of Large Language Models in Text-to-Image Synthesis Evaluation

PEViT

Official implementation of AAAI 2023 paper "Parameter-efficient Model Adaptation for Vision Transformers"

TIP

Multimodal-Procedural-Planning

Aerial-Vision-and-Dialog-Navigation

Codebase of ACL 2023 Findings "Aerial Vision-and-Dialog Navigation"

CPL

Official implementation of our EMNLP 2022 paper "CPL: Counterfactual Prompt Learning for Vision and Language Models"

FedVLN

[ECCV 2022] Official pytorch implementation of the paper "FedVLN: Privacy-preserving Federated Vision-and-Language Navigation"

T2IAT

T2IAT: Measuring Valence and Stereotypical Biases in Text-to-Image Generation

PECTVLM

Code implementation for Findings of EMNLP 2023 paper "Parameter-Efficient Cross-lingual Transfer of Vision and Language Models via Translation-based Alignment"

R2H

Official implementation of the EMNLP 2023 paper "R2H: Building Multimodal Navigation Helpers that Respond to Help Requests"

MultipanelVQA

Code for the MultipanelVQA benchmark