Situated Language and Embodied Dialogue (SLED) Research Group

chat-with-nerf

Chat with NeRF enables users to interact with a NeRF model by typing in natural language.

InfEdit

[CVPR 2024] Official implementation of CVPR 2024 paper: "Inversion-Free Image Editing with Natural Language"

CycleNet

Official Code for NeurIPS 2023 Paper: CycleNet: Rethinking Cycle Consistent in Text‑Guided Diffusion for Image Manipulation

MindCraft

Official code for our EMNLP2021 Outstanding Paper MindCraft: Theory of Mind Modeling for Situated Dialogue in Collaborative Tasks

DANLI

Code for EMNLP 2022 Paper DANLI: Deliberative Agent for Following Natural Language Instructions

DOROTHIE

Official Code for DOROTHIE: Spoken Dialogue for Handling Unexpected Situations in Interactive Autonomous Driving Agents (Findings of EMNLP 2022)

Verifiable-Coherent-NLU

Shared repository for TRIP dataset for verifiable NLU and coherence measurement for text classifiers.

world-to-words

Official Code for ACL 2023 Outstanding Paper: World-to-Words: Grounded Open Vocabulary Acquisition through Fast Mapping in Vision-Language Models

navchat

Code for ICRA24 paper "Think, Act, and Ask: Open-World Interactive Personalized Robot Navigation" https://arxiv.org/abs/2310.07968

Comparative-Learning

Official Code for ACL 2023 Paper: Human Inspired Progressive Alignment and Comparative Learning for Grounded Word Acquisition

Pragmatic-Rational-Speaker

Official code for ACL 2022 paper "Learning to Mediate Disparities Towards Pragmatic Communication".

collab-plan-acquisition

Official Code for IJCAI 2023 Paper: Towards Collaborative Plan Acquisition through Theory of Mind Modeling in Situated Dialogue

compute-guide

Guide to using compute resource at SLED Group

Watch-Talk-and-Guide

great-lakes-guide

Guide to Using Great Lakes

Heuristic-Analytic-Reasoning

Repo for the EMNLP 2023 paper "From Heuristic to Analytic: Cognitively-Motivated Reasoning Strategies for Coherent Physical Commonsense in Pre-Trained Language Models."

NLP-Reproducibility-For-Beginners

Repo for the ACL 2023 Paper "NLP Reproducibility For All: Understanding Experiences of Beginners."

Conversation-Entailment

Official dataset for Towards Conversation Entailment: An Empirical Investigation. Chen Zhang, Joyce Chai. EMNLP, 2010

Grounded-Semantic-Role-Labeling

Official dataset for Grounded Semantic Role Labeling. Shaohua Yang, Qiaozi Gao, Changsong Liu, Caiming Xiong, Song-Chun Zhu, Joyce Y. Chai. NAACL HLT, 2016.

HiTUT

Official code for the ACL 2021 Findings paper "Yichi Zhang and Joyce Chai. Hierarchical Task Learning from Language Instructions with Unified Transformers and Self-Monitoring"

Physical-Action-Effect-Prediction

Official dataset for "What Action Causes This? Towards Naive Physical Action-Effect Prediction", ACL 2018.

danli_visualizer

lerf

Code for LERF: Language Embedded Radiance Fields

lerf_lite

Code for LERF: Language Embedded Radiance Fields

nerfstudio_export

Nominal-Semantic-Role-Labeling

Official Annotation of our works on Nominal Semantic Role Labeling (NSRL)

PercepSync

Physical-Causality-of-Action-Verbs

Official dataset for Physical Causality of Action Verbs in Grounded Language Understanding. Qiaozi Gao, Malcolm Doering, Shaohua Yang, Joyce Chai. ACL, 2016.

Robust-Multimodal-Interpretation-in-Conversation-Systems

Official datasets for our works in Robust Multimodal Interpretation in Conversation Systems

TDW_Environment

Simulated Environment using TDW