vision-language-transformer

There are 5 repositories under vision-language-transformer topic.

salesforce / LAVIS
LAVIS - A One-stop Library for Language-Vision Intelligence
deep-learning deep-learning-library image-captioning multimodal-datasets multimodal-deep-learning salesforce vision-and-language vision-framework vision-language-pretraining vision-language-transformer visual-question-anwsering
Language:Jupyter Notebook 10889
IDEA-Research / GroundingDINO
[ECCV 2024] Official implementation of the paper "Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection"
object-detection open-world open-world-detection vision-language vision-language-transformer
Language:Python 8881
salesforce / BLIP
PyTorch code for BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
vision-language vision-and-language-pre-training image-text-retrieval image-captioning visual-question-answering visual-reasoning vision-language-transformer
Language:Jupyter Notebook 5172
AlibabaResearch / AdvancedLiterateMachinery
A collection of original, innovative ideas and algorithms towards Advanced Literate Machinery. This project is maintained by the OCR Team in the Language Technology Lab, Tongyi Lab, Alibaba Group.
artificial-intelligence documentai multimodal multimodal-deep-learning ocr computer-vision vision-language-transformer end-to-end-ocr scene-text-detection scene-text-detection-recognition scene-text-recognition text-detection text-recognition vision-language document document-analysis document-recognition document-understanding document-intelligence vision-language-model
Language:C++ 1769
henghuiding / ReLA
[CVPR2023 Highlight] GRES: Generalized Referring Expression Segmentation
multimodal-learning referring-expression-comprehension referring-expression-segmentation referring-image-segmentation vision-language-transformer cvpr2023
Language:Python 692
shenyunhang / APE
[CVPR 2024] Aligning and Prompting Everything All at Once for Universal Visual Perception
image-segmentation object-detection open-world referring-expression-comprehension vision-language-transformer
Language:Python 562
henghuiding / Vision-Language-Transformer
[ICCV2021 & TPAMI2023] Vision-Language Transformer and Query Generation for Referring Segmentation
iccv2021 keras referring-segmentation tensorflow tpami transformer vision-language vision-language-transformer
Language:Python 352
haoliuhl / instructrl
Instruction Following Agents with Multimodal Transforemrs
flax instruction-following instructions jax machine-learning reinforcement-learning transformer vision-language-transformer
Language:Python 52
sMamooler / CLIP_Explainability
code for studying OpenAI's CLIP explainability
computer-vision machine-learning openai-clip gradcam-visualization model-explainability vision-language-transformer
Language:Jupyter Notebook 30
yiren-jian / BLIText
[NeurIPS 2023] Bootstrapping Vision-Language Learning with Decoupled Language Pre-training
multimodal-deep-learning vision-language-pretraining vision-language-transformer
Language:Python 24
unitaryai / VTC
VTC: Improving Video-Text Retrieval with User Comments
multimodal-deep-learning video-text-retrieval video-understanding vision-language-pretraining vision-language-transformer comments
Language:Python 11
ThomasVonWu / Awesome-VLMs-Strawberry
A collection of VLMs papers, blogs, and projects, with a focus on VLMs in Autonomous Driving and related reasoning techniques.
llm multimodal-learning vision-language-transformer vlms
10
akusayudodograu / Agentic-RAG-Story-Generation-with-Multimodal-GenAI
Multimodal Agentic GenAI Workflow – Seamlessly blends retrieval and generation for intelligent storytelling
agentic-ai agentic-rag agentic-workflow generative-ai generative-ai-model internvl2 multimodal multimodal-deep-learning multimodal-large-language-models multimodal-learning story-generation vision-language vision-language-model vision-language-transformer
8
deepmancer / vlm-toolbox
Vision-Language Models Toolbox: Your all-in-one solution for multimodal research and experimentation
clip deep-learning deep-learning-library multimodal-datasets multimodal-deep-learning multimodal-learning prompt-tuning soft-prompt-tuning vision-and-language vision-framework vision-language-transformer zero-shot-classification hierarchical-classification multi-granularity-dataset pytorch transformers
Language:Jupyter Notebook 8
marialymperaiou / knowledge-enhanced-multimodal-learning
A list of research papers on knowledge-enhanced multimodal learning
conditional-image-generation image-text-matching image-text-retrieval knowledge-graph multimodal-deep-learning multimodal-retrieval story-visualization vision-and-language vision-and-language-navigation vision-and-language-pre-training vision-language-transformer visual-commonsense-reasoning visual-dialog visual-grounding visual-question-answering visual-reasoning visual-storytelling multi-task-learning knowledge-enhanced-multimodal-learning knowledge-enhanced-vision-language
7
fork123aniket / Multi-Round-VLM-powered-Multimodal-Conversational-AI-Navigation-Bot
Streamlit App Combining Vision, Language, and Audio AI Models
conversational-agent conversational-ai conversational-bot conversational-interface internvl internvl2 multimodal multimodal-data multimodal-deep-learning multimodal-large-language-models multimodal-learning vision-language vision-language-learning vision-language-model vision-language-models vision-language-navigation vision-language-transformer generative-ai
Language:Python 3
fork123aniket / Agentic-RAG-Story-Generation-with-Multimodal-GenAI
Multimodal Agentic GenAI Workflow – Seamlessly blends retrieval and generation for intelligent storytelling
agentic-ai agentic-rag agentic-workflow generative-ai internvl2 multimodal multimodal-data multimodal-deep-learning multimodal-large-language-models multimodal-learning story-generation vision-language vision-language-learning vision-language-model vision-language-transformer generative-ai-model
Language:Python 2
aurooj / VLM_SS
Mini-batch selective sampling for knowledge adaption of VLMs for mammography.
mammogram medical-imaging medical-vision-language-model miccai miccai2024 minibatch-selective-sampling multimodal-learning multimodal-representation-learning multimodal-retrieval vision-and-language vision-language-transformer
Language:Jupyter Notebook 1
PrateekJannu / Vision-GPT
Coding a Multi-Modal vision model like GPT-4o from scratch, inspired by @hkproj and PaliGemma
gemini google large-language-model large-language-models open-source transformer-architecture transformer-models vision-language-model vision-language-transformer vision-transformer artificial-intelligence gpt-4o llm machine-learning
Language:Python 1
atharva-naik / MMML-TermProject-VizWiz-VQA-Challenge
VizWiz Challenge Term Project for Multi Modal Machine Learning @ CMU (11777)
carnegie-mellon-university computer-vision image-processing natural-language-processing open-source open-source-project opencv pytorch question-answering term-project vision-language vision-language-transformer visual-question-answering vizwiz vizwiz-vqa
Language:Python 0