Cherryjingyao

visualnav-transformer

Official code and checkpoint release for mobile robot foundation models: GNM, ViNT, and NoMaD.

SuperPrompt

SuperPrompt is an attempt to engineer prompts that might help us understand AI agents.

Qwen-Agent

Agent framework and applications built upon Qwen2, featuring Function Calling, Code Interpreter, RAG, and Chrome extension.

Qwen2-VL

Qwen2-VL is the multimodal large language model series developed by Qwen team, Alibaba Cloud.

JARVIS-1

JARVIS-1: Open-world Multi-task Agents with Memory-Augmented Multimodal Language Models

Agent-Smith

[ICML2024] Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast

awesome-large-multimodal-agents

visualwebarena

VisualWebArena is a benchmark for multimodal agents.

Multi-Agent-GPT

Multi-Agent-GPT: 一款基于RAG和agent构建的多模态专家助手GPT。它集成了文本、图像和音频等模态工具。支持本地部署和私有数据库建设。

Chatglm_lora_multi-gpu

chatglm多gpu用deepspeed和

alfworld

ALFWorld: Aligning Text and Embodied Environments for Interactive Learning

JARVIS

JARVIS, a system to connect LLMs with ML community. Paper: https://arxiv.org/pdf/2303.17580.pdf

TaskMatrix

FireAct

FireAct: Toward Language Agent Fine-tuning

MiniCPM-V

MiniCPM-V 2.6: A GPT-4V Level MLLM for Single Image, Multi Image and Video on Your Phone

modelscope-agent

ModelScope-Agent: An agent framework connecting models in ModelScope with the world

autogen

A programming framework for agentic AI 🤖

agentUniverse

agentUniverse is a LLM multi-agent framework that allows developers to easily build multi-agent applications.

Sibyl-System

mem0

The Memory layer for your AI apps

Ask-Anything

[CVPR2024 Highlight][VideoChatGPT] ChatGPT with video understanding! And many more supported LMs such as miniGPT4, StableLM, and MOSS.

MiniGPT4-video

Official code for Goldfish model for long video understanding and MiniGPT4-video for short video understanding

humanplus

[CoRL 2024] HumanPlus: Humanoid Shadowing and Imitation from Humans

BLIP

PyTorch code for BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation

lamorel

Lamorel is a Python library designed for RL practitioners eager to use Large Language Models (LLMs).

ViLaIn

An official implementation of Vision-Language Interpreter (ViLaIn)

open_flamingo

An open-source framework for training large multimodal models.

VQASynth

Compose multimodal datasets 🎹

flamingo-pytorch

Implementation of 🦩 Flamingo, state-of-the-art few-shot visual question answering attention net out of Deepmind, in Pytorch

reflect

[CoRL 2023] REFLECT: Summarizing Robot Experiences for Failure Explanation and Correction