Conditional Neural Expert Processes (CNEP)

This repository contains the source code for the paper "Conditional Neural Expert Processes for Learning from Demonstration" by Yigit Yildirim and Emre Ugur. We both are members of the CoLoRs Lab, Bogazici University.

CNEP is a novel deep learning architecture for Learning from Demonstration (LfD) in robotics aiming to encode diverse sensorimotor trajectories from demonstrations with varying movements, leveraging a novel gating mechanism, multiple decoders, and an entropy-based loss calculation to promote decoder specialization. You can find the paper here: https://arxiv.org/abs/2402.08424

Architecture overview:

Some results:

Here are some videos from this work: https://youtube.com/playlist?list=PLXWw0F-8m_ZZD7fpGOKclzVJONXUifDiY&si=vR7OLhx_rigMxdLK

Getting Started

Requirements

The entire project is developed with

Python 3.8
Pytorch 2.0.1+cu117

However, most of the code should run cleanly with Python 3.8+ and Pytorch 2+. I tested with Python3.10 and several Pytorch 2+ versions.

Running

Clone the repo
Run the training script: for example, python -u compare_cnp_wta_sine.py
Upon training, run the corresponding test script: comparison_sine.ipynb
Naming convention: files starting with compare are training scripts whereas files starting with comparison are test scripts.
Files with the same name and different extensions:
1. Files with .ipynb extension are good for inspection & visualization
2. Files with .py extension are for running the code on a remote server (an HPC, for example).

If you use the code in your work, please cite https://arxiv.org/abs/2402.08424

zjq0717 / cnep