zhegan27

X-Decoder

[CVPR 2023] Official Implementation of X-Decoder for generalized decoding for pixel, image and language

GRiT

GRiT: A Generative Region-to-text Transformer for Object Understanding (https://arxiv.org/abs/2212.00280)

DDCap

FIBER

Coarse-to-Fine Vision-Language Pre-training with Fusion in the Backbone

GPT3-Reliability

mindall-e

PyTorch implementation of a 1.3B text-to-image generation model trained on 14 million image-text pairs

GenerativeImage2Text

GIT: A Generative Image-to-text Transformer for Vision and Language

UniTAB

UniTAB: Unifying Text and Box Outputs for Grounded VL Modeling, ECCV 2022 (Oral Presentation)

BEVT

PyTorch implementation of BEVT (CVPR 2022) https://arxiv.org/abs/2112.01529

ViT-Adapter

[ICLR 2023 Spotlight] Vision Transformer Adapter for Dense Predictions

SwinBERT

Research code for CVPR 2022 paper "SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioning"

SLIP

Code release for SLIP Self-supervision meets Language-Image Pre-training

DallEval

DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Models (ICCV 2023)

OFA

Official repository of OFA (ICML 2022). Paper: OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework

Detic

Code release for "Detecting Twenty-thousand Classes using Image-level Supervision".

BLIP

PyTorch code for BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation

pytorch_violet

A PyTorch implementation of VIOLET

ibot

iBOT :robot:: Image BERT Pre-Training with Online Tokenizer (ICLR 2022)

GLIP

Grounded Language-Image Pre-training

mlp-vil

MLPs for Vision and Langauge Modeling (Coming Soon)

METER

METER: A Multimodal End-to-end TransformER Framework

Stable-Pix2Seq

A full-fledged version of Pix2Seq

CV_A-FAN

[TMLR] "Adversarial Feature Augmentation and Normalization for Visual Recognition", Tianlong Chen, Yu Cheng, Zhe Gan, Jianfeng Wang, Lijuan Wang, Zhangyang Wang, Jingjing Liu

VL-T5

PyTorch code for "Unifying Vision-and-Language Tasks via Text Generation" (ICML 2021)

VidLanKD

Pytorch version of VidLanKD: Improving Language Understanding viaVideo-Distilled Knowledge Transfer (NeurIPS 2021))

P-tuning

A novel method to tune language models. Codes and datasets for paper ``GPT understands, too''.

ALBEF

Code for ALBEF: a new vision-language pre-training method

CoOp

Prompt Learning for Vision-Language Models (IJCV'22, CVPR'22)

compacter

Focal-Transformer

[NeurIPS 2021 Spotlight] Official code for "Focal Self-attention for Local-Global Interactions in Vision Transformers"