clownrat6

contrastors

Train Models Contrastively in Pytorch

open_clip

An open source implementation of CLIP.

RLAIF-V

RLAIF-V: Aligning MLLMs through Open-Source AI Feedback for Super GPT-4V Trustworthiness

SeaLLMs

[ACL 2024 Demo] SeaLLMs - Large Language Models for Southeast Asia

MT-LLaMA

Multi-Task instruction-tuned LLaMA

CER-MT

ConReader

CGR

code for "Exploiting Reasoning Chains for Multi-hop Science Question Answering"

AMR-SG

CUT

Source code of "Reasons to Reject? Aligning Language Models with Judgments"

OpenVid-1M

LivePortrait

Bring portraits to life!

Qwen2

Qwen2 is the large language model series developed by Qwen team, Alibaba Cloud.

AdaShield

[ECCV 2024] The official code for "AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Prompting."

datacomp

DataComp: In search of the next generation of multimodal datasets

ego4d-goalstep

Ego4D Goal-Step: Toward Hierarchical Understanding of Procedural Activities (NeurIPS 2023)

common_metrics_on_video_quality

You can easily calculate FVD, PSNR, SSIM, LPIPS for evaluating the quality of generated or predicted videos.

textgrad

TextGrad: Automatic ''Differentiation'' via Text -- using large language models to backpropagate textual gradients.

LongVA

Long Context Transfer from Language to Vision

VideoHallucer

VideoHallucer, The first comprehensive benchmark for hallucination detection in large video-language models (LVLMs)

tiny-diffusion

A minimal PyTorch implementation of probabilistic diffusion models for 2D datasets.

LLaVA-RLHF

Aligning LMMs with Factually Augmented RLHF

videollm-online

VideoLLM-online: Online Video Large Language Model for Streaming Video (CVPR 2024)

Math-LLaVA

Code for Math-LLaVA: Bootstrapping Mathematical Reasoning for Multimodal Large Language Models

VoCo-LLaMA

VoCo-LLaMA: This repo is the official implementation of "VoCo-LLaMA: Towards Vision Compression with Large Language Models".

Mantis

Official code for Paper "Mantis: Multi-Image Instruction Tuning"

WebDesignAgent

WebDesignAgent : Towards Effortless Website Creation

Recap-DataComp-1B

This is the official repository of our paper "What If We Recaption Billions of Web Images with LLaMA-3 ?"

medmcqa

A large-scale (194k), Multiple-Choice Question Answering (MCQA) dataset designed to address realworld medical entrance exam questions.

OmniTokenizer

OmniTokenizer: one model and one weight for image-video joint tokenization.