awesome-robot-imitation-learning-from-video
A collection of papers, codes and resources of learning from video in robotics.
Papers
Survey/Datasets
Recent Advances in Imitation Learning from Observation IJCAI 2019
Recent advances in robot learning from demonstration Annual Review of Control, Robotics, and Autonomous Systems 2020
What Matters in Learning from Offline Human Demonstrations for Robot Manipulation arxiv2021 | website |code
RoboTube: Learning Household Manipulation from Human Videos with Simulated Twin Environments RSS2022 | video
Challenges and Opportunities in Offline Reinforcement Learning from Visual Observations RSS 2022
Visual Representation Learning
R3M: A Universal Visual Representation for Robot Manipulation arXiv 2022 | code
Adversarial Skill Networks: Unsupervised Robot Skill Learning from Videos ICRA 2020 | website
Reinforcement Learning with Action-Free Pre-Training from Videos ICML2022 | code
Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos arXiv 2022
BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning CORL 2022 | website
VIP: TOWARDS UNIVERSAL VISUAL REWARD AND REPRESENTATION VIA VALUE-IMPLICIT PRETRAINING arXiv 2022
Estimated Pose/Key Point
DexVIP: Learning Dexterous Grasping with Human Hand Pose Priors from Video CORL 2021
DexMV: Imitation Learning for Dexterous Manipulation from Human Videos arxiv2022 | website | code
Learning Agile Robotic Locomotion Skills by Imitating Animals RSS 2020 | website | code
SFV: Reinforcement Learning of Physical Skills from Videos ACM Transactions on Graphics 2018 | code
Graph-Structured Visual Imitation CoRL 2019
Learning to Manipulate Tools by Aligning Simulation to Video Demonstration RAL 2022
Learning Periodic Tasks from Human Demonstrations arxiv2021
Robotic Telekinesis: Learning a Robotic Hand Imitator by Watching Humans on YouTube RSS 2022
You Only Demonstrate Once: Category-Level Manipulation from Single Visual Demonstration arxiv2022 VideoDex: Learning Dexterity from Internet Videos arxiv2022
Robotic Telekinesis: Learning a Robotic Hand Imitator by Watching Humans on YouTube arxiv2022
Graph Inverse Reinforcement Learning from Diverse Videos arxiv2022
Image Translation
Third-Person Visual Imitation Learning via Decoupled Hierarchical Controller NeurIPS 2019| website | code
Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation ICML 2016 | code
Learning by Watching: Physical Imitation of Manipulation Skills from Human Videos RSS 2021 | website
Meta Learning
One-Shot Hierarchical Imitation Learning of Compound Visuomotor Tasks arxiv 2018
One-shot imitation from observing humans via domain-adaptive meta-learning RSS 2018
Talks
Visual Imitation Learning: Generalization, Perceptual Grounding, and Abstraction RSS Workshop 2020
Imitation Learning Tutorial ICML 2018
Model-Based Visual Imitation Learning CVPR 2021 - Franziska Meier