Giters

AaronAnima / Diff4RLSurvey

This repository contains a collection of resources and papers on Diffusion Models for RL, accompanying the paper "Diffusion Models for Reinforcement Learning: A Survey"

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Diffusion Models for Reinforcement Learning: A Survey

This repository contains a collection of resources and papers on Diffusion Models for RL.

🚀 Please check out our survey paper Diffusion Models for Reinforcement Learning: A Survey

image info

Table of Contents

Papers

Offline Reinforcement Learning

  • Planning with Diffusion for Flexible Behavior Synthesis

    Michael Janner, Yilun Du, Joshua B. Tenenbaum, Sergey Levine

    ICML 2022

    paper / code

  • Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning

    Zhendong Wang, Jonathan J Hunt, Mingyuan Zhou

    ICLR 2023

    paper / code

  • Offline Reinforcement Learning via High-fidelity Generative Behavior Modeling

    Huayu Chen, Cheng Lu, Chengyang Ying, Hang Su, Jun Zhu

    ICLR 2023

    paper / code

  • Is Conditional Generative Modeling all you need for Decision-Making?

    Anurag Ajay, Yilun Du, Abhi Gupta, Joshua B. Tenenbaum, T. Jaakkola, Pulkit Agrawal

    ICLR 2023

    paper / code

  • AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Planners

    Zhixuan Liang, Yao Mu, Mingyu Ding, Fei Ni, Masayoshi Tomizuka, Ping Luo

    ICML 2023

    paper / code

  • Metadiffuser: Diffusion Model as Conditional Planner for Offline Meta-RL

    Fei Ni, Jianye Hao, Yao Mu, Yifu Yuan, Yan Zheng, Bin Wang, Zhixuan Liang

    ICML 2023

    paper

  • Hierarchical Diffusion for Offline Decision Making

    Wenhao Li, Xiangfeng Wang, Bo Jin, Hongyuan Zha.

    ICML 2023

    paper / code

  • Contrastive Energy Prediction for Exact Energy-guided Diffusion Sampling in Offline Reinforcement Learning

    Cheng Lu, Huayu Chen, Jianfei Chen, Hang Su, Chongxuan Li, Jun Zhu

    ICML 2023

    paper / code

  • Language Control Diffusion: Efficiently Scaling through Space, Time, and Tasks

    Edwin Zhang, Yujie Lu, William Wang, Amy Zhang

    arXiv 2023

    paper / code

  • IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policies

    Philippe Hansen-Estruch, Ilya Kostrikov, Michael Janner, Jakub Grudzien Kuba, Sergey Levine

    arXiv 2023

    paper / code

  • Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learning

    Haoran He, Chenjia Bai, Kang Xu, Zhuoran Yang, Weinan Zhang, Dong Wang, Bin Zhao, Xuelong Li

    NeurIPS 2023

    paper / code

  • EDGI: Equivariant Diffusion for Planning with Embodied Agents

    Johann Brehmer, Joey Bose, Pim de Haan, Taco Cohen

    NeurIPS 2023

    paper

  • Extracting Reward Functions from Diffusion Models

    Felipe Nuti, Tim Franzmeyer, João F. Henriques

    NeurIPS 2023

    paper

  • Can Pre-Trained Text-to-Image Models Generate Visual Goals for Reinforcement Learning?

    Jialu Gao, Kaizhe Hu, Guowei Xu, Huazhe Xu

    NeurIPS 2023

    paper

  • Reward-Directed Conditional Diffusion: Provable Distribution Estimation and Reward Improvement

    Hui Yuan, Kaixuan Huang, Chengzhuo Ni, Minshuo Chen, Mengdi Wang

    NeurIPS 2023

    paper / code

  • Refining Diffusion Planner for Reliable Behavior Synthesis by Automatic Detection of Infeasible Plans

    Kyowoon Lee, Seongun Kim, Jaesik Choi

    NeurIPS 2023

    paper / code

  • SafeDiffuser: Safe Planning with Diffusion Probabilistic Models

    Wei Xiao, Tsun-Hsuan Wang, Chuang Gan, Daniela Rus

    arXiv 2023

    paper

  • Efficient Diffusion Policies for Offline Reinforcement Learning

    Bingyi Kang, Xiao Ma, Chao Du, Tianyu Pang, Shuicheng Yan

    arXiv 2023

    paper / code

  • MADiff: Offline Multi-agent Learning with Diffusion Models

    Zhengbang Zhu, Minghuan Liu, Liyuan Mao, Bingyi Kang, Minkai Xu, Yong Yu, Stefano Ermon, Weinan Zhang

    arXiv 2023

    paper / code

  • Beyond Conservatism: Diffusion Policies in Offline Multi-agent Reinforcement Learning

    Zhuoran Li, Ling Pan, Longbo Huang

    arXiv 2023

    paper

  • Fighting Uncertainty with Gradients: Offline Reinforcement Learning via Diffusion Score Matching

    H.J. Terry Suh, Glen Chou, Hongkai Dai, Lujie Yang, Abhishek Gupta, Russ Tedrake

    CORL 2023

    paper / code

  • Value function estimation using conditional diffusion models for control

    Bogdan Mazoure, Walter Talbott, Miguel Angel Bautista, Devon Hjelm, Alexander Toshev, Josh Susskind

    arXiv 2023

    paper

  • Instructed Diffuser with Temporal Condition Guidance for Offline Reinforcement Learning

    Jifeng Hu, Yanchao Sun, Sili Huang, SiYuan Guo, Hechang Chen, Li Shen, Lichao Sun, Yi Chang, Dacheng Tao

    arXiv 2023

    paper

  • Diffusion Policies for Out-of-Distribution Generalization in Offline Reinforcement Learning

    Suzan Ece Ada, Erhan Oztop, Emre Ugur

    arXiv 2023

    paper

  • Diffusion Policies as Multi-Agent Reinforcement Learning Strategies

    Jinkun Geng, Xiubo Liang, Hongzhi Wang, Yu Zhao

    ICANN 2023

    paper

  • DiffCPS: Diffusion Model based Constrained Policy Search for Offline Reinforcement Learning

    Longxiang He, Linrui Zhang, Junbo Tan, Xueqian Wang

    arXiv 2023

    paper / code

  • Score Regularized Policy Optimization through Diffusion Behavior

    Huayu Chen, Cheng Lu, Zhengyi Wang, Hang Su, Jun Zhu

    arXiv 2023

    paper / code

  • Adaptive Online Replanning with Diffusion Models

    Siyuan Zhou, Yilun Du, Shun Zhang, Mengdi Xu, Yikang Shen, Wei Xiao, Dit-Yan Yeung, Chuang Gan

    arXiv 2023

    paper

  • AlignDiff: Aligning Diverse Human Preferences via Behavior-Customisable Diffusion Model

    Zibin Dong, Yifu Yuan, Jianye Hao, Fei Ni, Yao Mu, Yan Zheng, Yujing Hu, Tangjie Lv, Changjie Fan, Zhipeng Hu

    arXiv 2023

    paper / code

  • Boosting Continuous Control with Consistency Policy

    Yuhui Chen, Haoran Li, Dongbin Zhao

    arXIv 2023

    paper

Imitation Learning

  • Imitating Human Behaviour with Diffusion Models

    Tim Pearce, Tabish Rashid, Anssi Kanervisto, Dave Bignell, Mingfei Sun, Raluca Georgescu, Sergio Valcarcel Macua, Shan Zheng Tan, Ida Momennejad, Katja Hofmann, Sam Devlin**

    ICLR 2023

    paper / code

  • Diffusion Policy: Visuomotor Policy Learning via Action Diffusion

    Cheng Chi, Siyuan Feng, Yilun Du, Zhenjia Xu, Eric Cousineau, Benjamin Burchfiel, Shuran Song

    RSS 2023

    paper / code

  • Goal-Conditioned Imitation Learning using Score-based Diffusion Policies

    Moritz Reuss, Maximilian Li, Xiaogang Jia, Rudolf Lioutikov

    RSS 2023

    paper / code

  • To the Noise and Back: Diffusion for Shared Autonomy

    Takuma Yoneda, Luzhe Sun, and Ge Yang, Bradly Stadie, Matthew Walter

    RSS 2023

    paper / code

  • DALL-E-Bot: Introducing Web-Scale Diffusion Models to Robotics

    Ivan Kapelyukh, Vitalis Vosylius, Edward Johns

    RAL 2023

    paper

  • Scaling Up and Distilling Down: Language-Guided Robot Skill Acquisition

    Huy Ha, Pete Florence, Shuran Song

    CORL 2023

    paper / code

  • XSkill: Cross Embodiment Skill Discovery

    Mengda Xu, Zhenjia Xu, Cheng Chi, Manuela Veloso, Shuran Song

    CORL 2023

    paper

  • ChainedDiffuser: Unifying Trajectory Diffusion and Keypose Prediction for Robotic Manipulation

    Zhou Xian, Nikolaos Gkanatsios, Theophile Gervet, Tsung-Wei Ke, Katerina Fragkiadaki

    CORL 2023

    paper / code

  • PlayFusion: Skill Acquisition via Diffusion from Language-Annotated Play

    Lili Chen, Shikhar Bahl, Deepak Pathak

    CORL 2023

    paper

  • Generative Skill Chaining: Long-Horizon Skill Planning with Diffusion Models

    Utkarsh A. Mishra, Shangjie Xue, Yongxin Chen, Danfei Xu

    CORL 2023

    paper / code

  • Multimodal Diffusion Transformer for Learning from Play

    Moritz Reuss, Rudolf Lioutikov

    CORL 2023

    paper

  • GNFactor: Multi-Task Real Robot Learning with Generalizable Neural Feature Fields

    Yanjie Ze, Ge Yan, Yueh-Hua Wu, Annabella Macaluso, Yuying Ge, Jianglong Ye, Nicklas Hansen, Li Erran Li, Xiaolong Wang

    CORL 2023

    paper / code

  • Crossway Diffusion: Improving Diffusion-based Visuomotor Policy via Self-supervised Learning

    Xiang Li, Varun Belagali, Jinghuan Shang, Michael S. Ryoo

    arXiv 2023

    paper / code

  • Diffusion Co-Policy for Synergistic Human-Robot Collaborative Tasks

    Eley Ng, Ziang Liu, Monroe Kennedy III

    arXiv 2023

    paper / code

  • Compositional Foundation Models for Hierarchical Planning

    Anurag Ajay, Seungwook Han, Yilun Du, Shuang Li, Abhi Gupta, Tommi Jaakkola, Josh Tenenbaum, Leslie Kaelbling, Akash Srivastava, Pulkit Agrawal

    NeurIPS 2023

    paper / code

  • Generating Behaviorally Diverse Policies with Latent Diffusion Models

    Shashank Hegde, Sumeet Batra, K. R. Zentner, Gaurav S. Sukhatme

    NeurIPS 2023

    paper

  • NoMaD: Goal Masking Diffusion Policies for Navigation and Exploration

    Ajay Sridhar, Dhruv Shah, Catherine Glossop, Sergey Levine

    arXiv 2023

    paper / code

  • Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Models

    Kevin Black, Mitsuhiko Nakamoto, Pranav Atreya, Homer Walke, Chelsea Finn, Aviral Kumar, Sergey Levine

    arXiv 2023

    paper

Trajectory Generation

  • MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model

    Mingyuan Zhang, Zhongang Cai, Liang Pan, Fangzhou Hong, Xinying Guo, Lei Yang, Ziwei Liu

    arXiv 2022

    paper / code

  • Human Motion Diffusion Model

    Guy Tevet, Sigal Raab, Brian Gordon, Yonatan Shafir, Daniel Cohen-Or, Amit H. Bermano

    ICLR 2023

    paper / code

  • Executing your Commands via Motion Diffusion in Latent Space

    Xin Chen, Biao Jiang, Wen Liu, Zilong Huang, Bin Fu, Tao Chen, Jingyi Yu, Gang Yu

    CVPR 2023

    paper / code

  • MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesis

    Rishabh Dabral, Muhammad Hamza Mughal, Vladislav Golyanik, Christian Theobalt

    CVPR 2023

    paper / code

  • ReMoDiffuse: Retrieval-Augmented Motion Diffusion Model

    Mingyuan Zhang, Xinying Guo, Liang Pan, Zhongang Cai, Fangzhou Hong, Huirong Li, Lei Yang, Ziwei Liu

    ICCV 2023

    paper / code

  • MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusion

    Chiyu Max Jiang, Andre Cornman, Cheolho Park, Ben Sapp, Yin Zhou, Dragomir Anguelov

    CVPR 2023

    paper / code

  • Learning Universal Policies via Text-Guided Video Generation

    Yilun Du, Mengjiao Yang, Bo Dai, Hanjun Dai, Ofir Nachum, Joshua B. Tenenbaum, Dale Schuurmans, Pieter Abbeel

    NeurIPS 2023

    paper

  • EquiDiff: A Conditional Equivariant Diffusion Model For Trajectory Prediction

    Kehua Chen, Xianda Chen, Zihan Yu, Meixin Zhu, Hai Yang

    arXiv 2023

    paper

  • Motion Planning Diffusion: Learning and Planning of Robot Motions with Diffusion Models

    Joao Carvalho, An T. Le, Mark Baierl, Dorothea Koert, Jan Peters

    IROS 2023

    paper / code

  • EDMP: Ensemble-of-costs-guided Diffusion for Motion Planning

    Kallol Saha, Vishal Mandadi, Jayaram Reddy, Ajit Srikanth, Aditya Agarwal, Bipasha Sen, Arun Singh, Madhava Krishna

    arXiv 2023

    paper / code

  • Sampling Constrained Trajectories Using Composable Diffusion Models

    Thomas Power, Rana Soltani-Zarrin, Soshi Iba, Dmitry Berenson

    IROS 2023

    paper

  • DiMSam: Diffusion Models as Samplers for Task and Motion Planning under Partial Observability

    Xiaolin Fang, Caelan Reed Garrett, Clemens Eppner, Tomás Lozano-Pérez, Leslie Pack Kaelbling, Dieter Fox

    arXiv 2023

    paper

  • Conditioned Score-Based Models for Learning Collision-Free Trajectory Generation

    Joao Carvalho, Mark Baierl, Julen Urain, Jan Peters

    NeurIPSW 2022

    paper

  • Video Language Planning

    Yilun Du, Mengjiao Yang, Pete Florence, Fei Xia, Ayzaan Wahid, Brian Ichter, Pierre Sermanet, Tianhe Yu, Pieter Abbeel, Joshua B. Tenenbaum, Leslie Kaelbling, Andy Zeng, Jonathan Tompson

    arXiv 2023

    paper / code

  • Learning to Act from Actionless Video through Dense Correspondences

    Po-Chen Ko, Jiayuan Mao, Yilun Du, Shao-Hua Sun, Joshua B. Tenenbaum

    arXiv 2023

    paper / code

  • Learning Interactive Real-World Simulators

    Mengjiao Yang, Yilun Du, Kamyar Ghasemipour, Jonathan Tompson, Dale Schuurmans, Pieter Abbeel

    arXiv 2023

    paper

Data Augmentation

  • Scaling Robot Learning with Semantically Imagined Experience

    Tianhe Yu, Ted Xiao, Austin Stone, Jonathan Tompson, Anthony Brohan, Su Wang, Jaspiar Singh, Clayton Tan, Dee M, Jodilyn Peralta, Brian Ichter, Karol Hausman, Fei Xia

    RSS 2023

    paper

  • GenAug: Retargeting behaviors to unseen situations via Generative Augmentation

    Zoey Chen, Sho Kiami, Abhishek Gupta, Vikash Kumar

    RSS 2023

    paper / code

  • Synthetic Experience Replay

    Cong Lu, Philip J. Ball, Yee Whye Teh, Jack Parker-Holder

    NeurIPS 2023

    paper / code

Citation

@article{zhu2023diffusion,
  title={Diffusion Models for Reinforcement Learning: A Survey},
  author={Zhu, Zhengbang and Zhao, Hanye and He, Haoran and Zhong, Yichao and Zhang, Shenyu and Yu, Yong and Zhang, Weinan},
  journal={arXiv preprint arXiv:2311.01223},
  year={2023}
}

About

This repository contains a collection of resources and papers on Diffusion Models for RL, accompanying the paper "Diffusion Models for Reinforcement Learning: A Survey"

License:Apache License 2.0