vision-and-language

There are 26 repositories under vision-and-language topic.

salesforce / LAVIS
LAVIS - A One-stop Library for Language-Vision Intelligence
deep-learning deep-learning-library image-captioning salesforce vision-and-language vision-framework vision-language-pretraining vision-language-transformer visual-question-anwsering multimodal-datasets multimodal-deep-learning
Language:Jupyter Notebook 9268
aishwaryanr / awesome-generative-ai-guide
A one stop repository for generative AI research updates, interview resources, notebooks and much more!
awesome awesome-list generative-ai interview-questions large-language-models llms notebook-jupyter vision-and-language
6352
salesforce / ALBEF
Code for ALBEF: a new vision-language pre-training method
contrastive-learning image-text representation-learning vision-and-language weakly-supervised-learning
Language:Python 1462
open-mmlab / Multimodal-GPT
Multimodal-GPT
flamingo gpt gpt-4 llama multimodal transformer vision-and-language
Language:Python 1444
om-ai-lab / OmDet
Real-time and accurate open-vocabulary end-to-end object detection
object-detection open-vocabulary vision-and-language zero-shot-object-detection
Language:Python 1368
dandelin / ViLT
Code for the ICML 2021 (long talk) paper: "ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision"
vision-and-language
Language:Python 1337
NVlabs / prismer
The implementation of "Prismer: A Vision-Language Model with Multi-Task Experts".
image-captioning language-model multi-modal-learning multi-task-learning vision-language-model vision-and-language vqa
Language:Python 1292
yuewang-cuhk / awesome-vision-language-pretraining-papers
Recent Advances in Vision and Language PreTrained Models (VL-PTMs)
bert multimodal-deep-learning pretraining vision-and-language vl-ptms
1133
Oscar
microsoft / Oscar
Oscar and VinVL
image-captioning image-text-search oscar pre-training vinvl vision-and-language vqa
Language:Python 1034
YehLi / xmodaler
X-modaler is a versatile and high-performance codebase for cross-modal analytics(e.g., image captioning, video captioning, vision-language pre-training, visual question answering, visual commonsense reasoning, and cross-modal retrieval).
image-captioning video-captioning vision-and-language pretraining cross-modal-retrieval visual-question-answering tden
Language:Python 1017
OFA-Sys / ONE-PEACE
A general representation model across vision, audio, language modalities. Paper: ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities
foundation-models multimodal representation-learning vision-language audio-language vision-and-language vision-transformer contrastive-loss
Language:Python 898
llm-jp / awesome-japanese-llm
日本語LLMまとめ - Overview of Japanese LLMs
foundation-models generative-ai generative-model generative-models japanese japanese-language japanese-language-model japanese-llm language-model language-models large-language-model large-language-models llm llm-japanese llms multimodal vision-and-language vision-language vision-language-model
Language:TypeScript 881
26hzhang / DL-NLP-Readings
My Reading Lists of Deep Learning and Natural Language Processing
paper source-code-link deep-learning natural-language-processing reinforcement-learning commonsense language-model robotics machine-learning bibtex-references vision-and-language
Language:TeX 850
ChenRocks / UNITER
Research code for ECCV 2020 paper "UNITER: UNiversal Image-TExt Representation Learning"
pre-training pytorch transformers vision-and-language
Language:Python 772
jackroos / VL-BERT
Code for ICLR 2020 paper "VL-BERT: Pre-training of Generic Visual-Linguistic Representations".
bert iclr2020 pre-training pytorch representation-learning self-supervised-learning vision-and-language vl-bert
Language:Jupyter Notebook 735
jayleicn / ClipBERT
[CVPR 2021 Best Student Paper Honorable Mention, Oral] Official PyTorch code for ClipBERT, an efficient framework for end-to-end learning on image-text and video-text tasks.
pytorch video-retrieval video-question-answering vqa vision-and-language cvpr2021
Language:Python 693
mbzuai-oryx / groundingLMM
[CVPR 2024 🔥] Grounding Large Multimodal Model (GLaMM), the first-of-its-kind model capable of generating natural language responses that are seamlessly integrated with object segmentation masks.
foundation-models llm-agent lmm vision-and-language vision-language-model
Language:Python 689
SkalskiP / top-cvpr-2023-papers
This repository is a curated collection of the most exciting and influential CVPR 2023 papers. 🔥 [Paper + Code]
computer-vision cvpr cvpr2023 image-segmentation object-detection paper transformers vision-and-language
Language:Python 636
SkalskiP / top-cvpr-2024-papers
This repository is a curated collection of the most exciting and influential CVPR 2024 papers. 🔥 [Paper + Code + Demo]
computer-vision cvpr cvpr2024 image-segmentation object-detection paper transformers vision-and-language
Language:Python 590
SunzeY / AlphaCLIP
[CVPR 2024] Alpha-CLIP: A CLIP Model Focusing on Wherever You Want
deep-learning machine-learning vision-language vision-language-model vision-transformer vision-and-language
Language:Jupyter Notebook 590
vardanagarwal / Proctoring-AI
Creating a software for automatic monitoring in online proctoring
yolov3 dlib proctoring proctoring-ai opencv nltk vision-and-language speech-to-text automation face-spoofing face-detection phone-detection eye-tracking mobilenet tflite ssd hacktoberfest
Language:Python 538
NVlabs / DoRA
[ICML2024 (Oral)] Official PyTorch implementation of DoRA: Weight-Decomposed Low-Rank Adaptation
commonsense-reasoning deep-learning deep-neural-networks instruction-tuning large-language-models large-vision-language-models lora parameter-efficient-fine-tuning parameter-efficient-tuning vision-and-language
Language:Python 471
peteanderson80 / Matterport3DSimulator
AI Research Platform for Reinforcement Learning from Real Panoramic Images.
rl simulator matterport3d-dataset matterport3d-simulator reinforcement-learning vision-and-language natural-language-processing
Language:C++ 471
OpenRobotLab / PointLLM
[ECCV 2024] PointLLM: Empowering Large Language Models to Understand Point Clouds
3d chatbot foundation-models gpt-4 large-language-models llama multimodal objaverse point-cloud pointllm representation-learning vision-and-language
Language:Python 443
zengyan-97 / X-VLM
X-VLM: Multi-Grained Vision Language Pre-Training (ICML 2022)
x-vlm multimodality vision-and-language
Language:Python 434
sangminwoo / awesome-vision-and-language
A curated list of awesome vision and language resources (still under construction... stay tuned!)
awesome awesome-list vision-and-language multimodal-learning
416
Paranioar / Awesome_Matching_Pretraining_Transfering
The Paper List of Large Multi-Modality Model, Parameter-Efficient Finetuning, Vision-Language Pretraining, Conventional Image-Text Matching for Preliminary Insight.
awesome-list cross-modal-retrieval image-text-matching image-text-retrieval large-language-models large-vision-language-models large-vision-models memory-efficient-tuning multimodal-pretraining parameter-efficient-fine-tuning tutorial video-text-recognition video-text-retrieval vision-and-language visual-semantic-embedding
375
j-min / VL-T5
PyTorch code for "Unifying Vision-and-Language Tasks via Text Generation" (ICML 2021)
pretraining transformers vision-and-language vl-bart vl-t5
Language:Python 355
tsujuifu / pytorch_mgie
A Gradio demo of MGIE
iclr2024 image-editing multimodal-large-language-models pytorch vision-and-language
Language:Python 339
google-research-datasets / conceptual-12m
Conceptual 12M is a dataset containing (image-URL, caption) pairs collected for vision-and-language pre-training.
vision-and-language pre-training multimodal-dataset
338
mees / calvin
CALVIN - A benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks
computer-vision deep-learning grounding manipulation natural-language-processing pytorch robotics vision vision-and-language vision-language
Language:Python 321
JindongGu / Awesome-Prompting-on-Vision-Language-Model
This repo lists relevant papers summarized in our survey paper: A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models.
foundation-models prompt-engineering vision-and-language
315
eric-ai-lab / awesome-vision-language-navigation
A curated list for vision-and-language navigation. ACL 2022 paper "Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directions"
vision-and-language-navigation vision-and-language navigation embodied-agent
310
HyperGAI / HPT
HPT - Open Multimodal LLMs from HyperGAI
generative-ai multimodal vision-and-language
Language:Python 303
phellonchen / awesome-Vision-and-Language-Pre-training
Recent Advances in Vision and Language Pre-training (VLP)
vision-and-language-pre-training vision-and-language pretraining multimodal-deep-learning vlp
283
JDAI-CV / image-captioning
Implementation of 'X-Linear Attention Networks for Image Captioning' [CVPR 2020]
image-captioning vision-and-language
Language:Python 269

vision-and-language

salesforce / LAVIS

aishwaryanr / awesome-generative-ai-guide

salesforce / ALBEF

open-mmlab / Multimodal-GPT

om-ai-lab / OmDet

dandelin / ViLT

NVlabs / prismer

yuewang-cuhk / awesome-vision-language-pretraining-papers

microsoft / Oscar

YehLi / xmodaler

OFA-Sys / ONE-PEACE

llm-jp / awesome-japanese-llm

26hzhang / DL-NLP-Readings

ChenRocks / UNITER

jackroos / VL-BERT

jayleicn / ClipBERT

mbzuai-oryx / groundingLMM

SkalskiP / top-cvpr-2023-papers

SkalskiP / top-cvpr-2024-papers

SunzeY / AlphaCLIP

vardanagarwal / Proctoring-AI

NVlabs / DoRA

peteanderson80 / Matterport3DSimulator

OpenRobotLab / PointLLM

zengyan-97 / X-VLM

sangminwoo / awesome-vision-and-language

Paranioar / Awesome_Matching_Pretraining_Transfering

j-min / VL-T5

tsujuifu / pytorch_mgie

google-research-datasets / conceptual-12m

mees / calvin

JindongGu / Awesome-Prompting-on-Vision-Language-Model

eric-ai-lab / awesome-vision-language-navigation

HyperGAI / HPT

phellonchen / awesome-Vision-and-Language-Pre-training

JDAI-CV / image-captioning