Ferenas

segment-anything-2

The repository provides code for running inference with the Meta Segment Anything Model 2 (SAM 2), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.

ImageBind

ImageBind One Embedding Space to Bind Them All

open_llama

OpenLLaMA, a permissively licensed open source reproduction of Meta AI’s LLaMA 7B trained on the RedPajama dataset

mae

PyTorch implementation of MAE https//arxiv.org/abs/2111.06377

AnyText

Official implementation code of the paper <AnyText: Multilingual Visual Text Generation And Editing>

NExT-GPT

Code and models for NExT-GPT: Any-to-Any Multimodal Large Language Model

VILA

VILA - a multi-image visual language model with training, inference and evaluation recipe, deployable from cloud to edge (Jetson Orin and laptops)

self-rag

This includes the original implementation of SELF-RAG: Learning to Retrieve, Generate and Critique through self-reflection by Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi.

ml-4m

4M: Massively Multimodal Masked Modeling

AdvancedLiterateMachinery

A collection of original, innovative ideas and algorithms towards Advanced Literate Machinery. This project is maintained by the OCR Team in the Language Technology Lab, Tongyi Lab, Alibaba Group.

LlamaGen

Autoregressive Model Beats Diffusion: 🦙 Llama for Scalable Image Generation

Qwen2-Audio

The official repo of Qwen2-Audio chat & pretrained large audio language model proposed by Alibaba Cloud.

anole

Anole: An Open, Autoregressive and Native Multimodal Models for Interleaved Image-Text Generation

Curve-Text-Detector

This repository provides train＆test code, dataset, det.&rec. annotation, evaluation script, annotation tool, and ranking.

OneLLM

[CVPR 2024] OneLLM: One Framework to Align All Modalities with Language

unified-io-2

awesome-large-audio-models

Collection of resources on the applications of Large Language Models (LLMs) in Audio AI.

Awesome-LLMs-meet-Multimodal-Generation

🔥🔥🔥 A curated list of papers on LLMs-based multimodal generation (image, video, 3D and audio).

Anything2Image

Generate image from anything with ImageBind and Stable Diffusion

ICV

Code for In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering

Seeing-and-Hearing

[CVPR 2024] Seeing and Hearing: Open-domain Visual-Audio Generation with Diffusion Latent Aligners

EasyComDataset

The Easy Communications (EasyCom) dataset is a world-first dataset designed to help mitigate the *cocktail party effect* from an augmented-reality (AR) -motivated multi-sensor egocentric world view.

VQGAN-LC

VQ-VAE

Pytorch Implementation of "Neural Discrete Representation Learning"

Localizing-Visual-Sounds-the-Hard-Way

Localizing Visual Sounds the Hard Way

icl_task_vectors

garnet

E2STR

The official code for the CVPR 2024 paper: Multi-modal In-Context Learning Makes an Ego-evolving Scene Text Recognizer

VL-ICL

Code for paper: VL-ICL Bench: The Devil in the Details of Benchmarking Multimodal In-Context Learning

Awesome-Autoregressive-Visual-Generation

This is a repo to track the latest autoregressive visual generation papers.