Composable Agent Toolbox (CAT)
Composable benchmark for intelligent agents. The design can support
- modularized agents and end-to-end agents, and
- model-based agents and model-free agents.
Model
The model module contains structured differentiable functions that either represent system dynamics or control policies. The model can be analytical (i.e., ODEs, basis functions) or numerical (i.e., neural networks). For now, we implemented analytical models for system dyanmics, which are encoded symbolically. Both linear/nonlinear systems are supported.
Planner
The planner module contains a class of functions that map observation to a sequence of future states of future inputs. A planner can either be model-based or model-free. For now, we implemented model-based planners, especially optimization-based planners.
Estimator
The estimator module contains a class of functions that map loss to parameter update signals. The loss can be prediction errors. The objects being estimated can either be states or parameters (either in the dynamics or in the controller/planner). These states or parameters can either from the ego agent or other agents in the environment. When we estimate the parameters, it is viewed as a learning process. The algorithm for the estimation can either be first order, e.g., SGD, or second order, e.g., RLS, KF, EKF, UKF. For now, we implemented second order estimation methods for system states.
Controller
The controller module contains a class of functions that map observation to the next control input. The only difference between a planner and a controller lies on their outputs, where the planner outputs a sequence which contains multiple future actions, while the controller outputs one future action. A controller can either be model-based or model-free. For now, we implemented model-based controllers, e.g., tracking control, safe control.
Agents
Hierarchical Agents
An agent that runs planner and controller at different frequencies.
End-to-End Agents
An agent that use an end-to-end black-box policy.
Requirements
We specified the dependencies in requirements.txt
. You can reproduce the environment by
conda create -n CAT python=3.8
conda activate CAT
cd $REPO_PATH
pip install -r requirements.txt
If python raises error of graphviz, try reinstalling graphviz with either of the following commands
pip install graphviz
conda install python-graphviz
brew install graphviz
sudo apt install graphviz
If you want to use the Mujoco env. Install mujoco_py with
pip install mujoco_py==2.0.2.8
Usage
See examples for more information.
DSTA
DONE
- General
- Now computation agent spec is read from env yaml (as file path) instead of coding in py
- Planning Models
- Double integrator model
- Planner
- Naive planner (interpolation, does not use model)
- Integrator planner (needs double/triple/etc. integrator model)
- Control Models
- Ballmodel
- Unicycle
- Non linear PID
- Feedback Controller
- Naive (use arbitrary control models)
Assumption
- Planner takes goal + goal type defined by task
- Controller will always use full dimension of planner traj, but can ignore time deritatives
- Control model should be consistent with/same as that of world agent.
- The world agent needs to provid all info needed by control model
HW1 Safe Control
HW2 Safe Planning
python flat_reach.py
Note
examples/configs/flat_reach_agent_2.yaml
usesCFSPlanner
defined inagent/planner/Planner.py
which calls_CFS()
to solve planning.- Planner is invoked at
agent/model_based_agent.py:51
.