visual-question-answering

There are 19 repositories under visual-question-answering topic.

salesforce / BLIP
PyTorch code for BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
image-captioning image-text-retrieval vision-and-language-pre-training vision-language vision-language-transformer visual-question-answering visual-reasoning
Language:Jupyter Notebook 4338
OFA-Sys / OFA
Official repository of OFA (ICML 2022). Paper: OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework
chinese image-captioning multimodal pretrained-models pretraining prompt prompt-tuning referring-expression-comprehension text-to-image-synthesis vision-language visual-question-answering
Language:Python 2340
peteanderson80 / bottom-up-attention
Bottom-up attention model for image captioning and VQA, based on Faster R-CNN and Visual Genome
caffe captioning-images faster-rcnn image-captioning mscoco mscoco-dataset visual-question-answering vqa
Language:Jupyter Notebook 1409
lucidrains / flamingo-pytorch
Implementation of 🦩 Flamingo, state-of-the-art few-shot visual question answering attention net out of Deepmind, in Pytorch
artificial-intelligence attention-mechanism deep-learning transformers visual-question-answering
Language:Python 1153
YehLi / xmodaler
X-modaler is a versatile and high-performance codebase for cross-modal analytics(e.g., image captioning, video captioning, vision-language pre-training, visual question answering, visual commonsense reasoning, and cross-modal retrieval).
cross-modal-retrieval image-captioning pretraining tden video-captioning vision-and-language visual-question-answering
Language:Python 1011
jnhwkim / ban-vqa
Bilinear attention networks for visual question answering
attention bilinear-pooling pytorch-implmention visual-question-answering
Language:Python 536
MILVLG / mcan-vqa
Deep Modular Co-Attention Networks for Visual Question Answering
attention visual-question-answering visual-reasoning
Language:Python 432
davidmascharka / tbd-nets
PyTorch implementation of "Transparency by Design: Closing the Gap Between Performance and Interpretability in Visual Reasoning"
machine-learning pytorch visualization deep-learning visual-question-answering vqa neural-networks
Language:Jupyter Notebook 348
richard-peng-xia / awesome-multimodal-in-medical-imaging
A collection of resources on applications of multi-modal learning in medical imaging.
medical-imaging medical-report-generation multimodal-deep-learning multimodal-learning visual-question-answering large-language-models large-multimodal-models multimodal-large-language-models
343
MILVLG / openvqa
A lightweight, scalable, and general framework for visual question answering research
benchmark deep-learning pytorch visual-question-answering vqa
Language:Python 309
MMMU-Benchmark / MMMU
This repo contains evaluation code for the paper "MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI"
computer-vision deep-learning deep-neural-networks evaluation foundation-models large-language-models large-multimodal-models llm llms machine-learning multimodal multimodal-deep-learning multimodal-learning multimodality natural-language-processing question-answering stem visual-question-answering
Language:Python 263
MILVLG / prophet
Implementation of CVPR 2023 paper "Prompting Large Language Models with Answer Heuristics for Knowledge-based Visual Question Answering".
a-okvqa gpt-3 multimodal-deep-learning okvqa prompt-engineering pytorch visual-question-answering
Language:Python 261
Cyanogenoid / pytorch-vqa
Strong baseline for visual question answering
pytorch vqa visual-question-answering baseline
Language:Python 239
zjukg / KG-MM-Survey
Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Survey
awsome-list cross-modal-retrieval entity-alignment entity-linking image-classification image-generation information-extraction knowledge-graph knowledge-graph-embeddings large-language-models multi-modal-fusion multi-modal-knowledge-graph multi-modal-learning paper-list survey surveys visual-question-answering awsome
209
lupantech / MathVista
MathVista: data, code, and evaluation for Mathematical Reasoning in Visual Contexts
ai4math large-language-models large-multimadality-models machine-learning mathematics mathqa science visual-question-answering
Language:Jupyter Notebook 187
markdtw / vqa-winner-cvprw-2017
Pytorch implementation of winner from VQA Chllange Workshop in CVPR'17
pytorch visual-question-answering
Language:Python 165
antoyang / FrozenBiLM
[NeurIPS 2022] Zero-Shot Video Question Answering via Frozen Bidirectional Language Models
multimodal-learning video-understanding vqa weakly-supervised-learning large-language-models pre-training video-question-answering videoqa vision-and-language visual-question-answering
Language:Python 144
qiantianwen / NuScenes-QA
[AAAI 2024] NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenario.
autonomous-driving vision-language visual-question-answering
130
zhegan27 / VILLA
Research Code for NeurIPS 2020 Spotlight paper "Large-Scale Adversarial Training for Vision-and-Language Representation Learning": UNITER adversarial training part
vision-and-language adversarial-training pretraining visual-question-answering neurips-2020
Language:Python 118
just-ask
antoyang / just-ask
[ICCV 2021 Oral + TPAMI] Just Ask: Learning to Answer Questions from Millions of Narrated Videos
vqa visual-question-answering videoqa video-question-answering video-understanding question-generation weakly-supervised-learning vision-and-language pre-training multimodal-learning
Language:Jupyter Notebook 114
Yushi-Hu / tifa
TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering
image-to-text large-language-models text-to-image visual-question-answering
Language:Python 114
MMStar-Benchmark / MMStar
This repo contains evaluation code for the paper "Are We on the Right Way for Evaluating Large Vision-Language Models"
evaluation large-language-models large-multimodal-models large-vision-language-model large-vision-language-models llm llms lvlm lvlms multimodal multimodal-learning multimodality visual-question-answering
Language:Python 110
anisha2102 / docvqa
Document Visual Question Answering
computer-vision deep-learning document-analysis visual-question-answering
Language:Python 105
HanXinzi-AI / awesome-computer-vision-resources
a collection of computer vision projects&tools. 计算机视觉方向项目和工具集合。
autonomous-driving autonomous-vehicles computer-vision face-detection face-recognition gan image-captioning image-classification image-segmentation medical-imaging model-compression model-optimization ocr paddlepaddle pedestrian-detection pytorch semantic-segmentation super-resolution tensorflow visual-question-answering
101
mesnico / RelationNetworks-CLEVR
A pytorch implementation for "A simple neural network module for relational reasoning", working on the CLEVR dataset
relation-network relationships clevr deep-learning machine-learning visual-question-answering pytorch
Language:Python 85
sdc17 / UPop
[ICML 2023] UPop: Unified and Progressive Pruning for Compressing Vision-Language Transformers.
efficient-deep-learning model-compression multimodal-learning vision-language-transformer image-captioning image-text-retrieval visual-question-answering visual-reasoning text-image-retrieval framework weight-pruning sparsity structured vision-transformer pruning
Language:Python 85
Shivanshu-Gupta / Visual-Question-Answering
CNN+LSTM, Attention based, and MUTAN-based models for Visual Question Answering
attention cnn deep-learning lstm multimodal-tucker-fusion nlp pytorch stacked-attention-networks visual-question-answering
Language:Python 71
violetteshev / bottom-up-features
Bottom-up features extractor implemented in PyTorch.
vqa visual-question-answering faster-rcnn feature-extraction pytorch
Language:Python 71
rentainhe / TRAR-VQA
[ICCV 2021] Official implementation of the paper "TRAR: Routing the Attention Spans in Transformers for Visual Question Answering"
vqav2 iccv2021 transformer clevr multi-modal vision-and-language visual-question-answering pytorch multi-scale-features dynamic-network attention local-and-global multi-modality visualization multi-modal-learning official
Language:Python 61
China-UK-ZSL / ZS-F-VQA
[Paper][ISWC 2021] Zero-shot Visual Question Answering using Knowledge Graph
vqa zero-shot knowledge-graph commonsense commonsense-reasoning visual-question-answering zsl fvqa zs-f-vqa
Language:Python 60
DenisDsh / VizWiz-VQA-PyTorch
PyTorch VQA implementation that achieved top performances in the (ECCV18) VizWiz Grand Challenge: Answering Visual Questions from Blind People
vizwiz vqa pytorch visual-question-answering
Language:Jupyter Notebook 59
mlvlab / Flipped-VQA
Large Language Models are Temporal and Causal Reasoners for Video Question Answering (EMNLP 2023)
emnlp2023 large-language-models multi-modal video-question-answering visual-question-answering
Language:Python 57
badripatro / PQG
Code for paper title "Learning Semantic Sentence Embeddings using Pair-wise Discriminator" COLING-2018
vqa vqg questions-and-answers question-answering question-generation question-parapharse visual-questions-generation visual-question-answering coling2018 emnlp2018 emnlp2017 emnlp acl acl-2018 paraphrase paraphrase-generation paraphrase-identification torch bleu rouge
Language:Jupyter Notebook 54
KVQA
SKTBrain / KVQA
Korean Visual Question Answering
visual-question-answering dataset korean
53
allenai / aokvqa
Official repository for the A-OKVQA dataset
computer-vision dataset natural-language-processing visual-question-answering
Language:Python 50
caffeinism / FiLM-pytorch
PyTorch implementation of FiLM: Visual Reasoning with a General Conditioning Layer
visual-question-answering
Language:Python 49