visual-language-learning

There are 5 repositories under visual-language-learning topic.

haotian-liu / LLaVA
[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.
gpt-4 chatbot chatgpt llama multimodal llava foundation-models instruction-tuning multi-modality visual-language-learning llama-2 llama2 vision-language-model
Language:Python 23912
NExT-GPT / NExT-GPT
Code and models for ICML 2024 paper, NExT-GPT: Any-to-Any Multimodal Large Language Model
chatgpt foundation-models gpt-4 instruction-tuning large-language-models llm mllm multi-modal-chatgpt multimodal visual-language-learning
Language:Python 3579
Otter
EvolvingLMMs-Lab / Otter
🦦 Otter, a multi-modal model based on OpenFlamingo (open-sourced version of DeepMind's Flamingo), trained on MIMIC-IT and showcasing improved instruction-following and in-context learning ability.
artificial-inteligence chatgpt deep-learning embodied-ai foundation-models gpt-4 instruction-tuning large-scale-models machine-learning multi-modality visual-language-learning
Language:Python 3276
InternLM / InternLM-XComposer
InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions
chatgpt foundation gpt gpt-4 instruction-tuning language-model large-language-model large-vision-language-model llm mllm multi-modality multimodal supervised-finetuning vision-language-model vision-transformer visual-language-learning
Language:Python 2899
xiaoachen98 / Open-LLaVA-NeXT
An open-source implementation for training LLaVA-NeXT.
chatbot chatgpt gpt-4 gpt4o large-multimodal-models llama llama3 llava multi-modality multimodal vision-language-model visual-language-learning llava-next
Language:Python 425
RLHF-V / RLHF-V
[CVPR'24] RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback
chatbot gpt-4 llama multi-modality multimodal rlhf-v visual-language-learning
Language:Python 295
mlpc-ucsd / BLIVA
(AAAI 2024) BLIVA: A Simple Multimodal LLM for Better Handling of Text-rich Visual Questions
blip2 bliva chatbot instruction-tuning llama llm lora multimodal visual-language-learning
Language:Python 260
thomas-yanxin / KarmaVLM
🧘🏻‍♂️KarmaVLM (相生)：A family of high efficiency and powerful visual language model.
llama2 llava qwen2 vlm vision-language-model visual-language-learning multimodel
Language:Python 88
AdrianBZG / llama-multimodal-vqa
Multimodal Instruction Tuning for Llama 3
chatbot chatgpt gpt-4 huggingface instruction-tuning language-models llama llama2 llama3 multimodal multimodal-instruction-tuning visual-language-learning visual-question-answering vqa
Language:Python 50
xinyanghuang7 / Basic-Visual-Language-Model
Build a simple basic multimodal large model from scratch. 从零搭建一个简单的基础多模态大模型🤖
large-language-models multimodel-large-language-model visual-language-learning visual-language-models
Language:Python 47
Skyline-9 / Shotluck-Holmes
[ACM MMGR '24] 🔍 Shotluck Holmes: A family of small-scale LLVMs for shot-level video understanding
llm nlp python video-captioning video-summarization multi-modality vision-language-model visual-language-learning shotluck-holmes
Language:Python 13
ashleykleynhans / llava-docker
Docker image for LLaVA: Large Language and Vision Assistant
ai chatbot chatgpt docker docker-image foundation-models gpt-4 instruction-tuning llama llama-2 llama2 llava llm multimodal runpod vision-language-model visual-language-learning
Language:Shell 4
MuhammadAliS / CLIP
PyTorch implementation of OpenAI's CLIP model for image classification, visual search, and visual question answering (VQA).
deep-neural-networks huggingface pytorch-implementation transformers visual-language-learning visual-question-answering
Language:Jupyter Notebook 3
ecoxial2007 / EffVideoQA
Efficient Video Question Answering
computer-vision video-question-answering visual-language-learning
Language:Python 1