Question: SubTasks Timestepping

[Peter David Fagan]

Background

I'm a PhD student and I'm trying to use MuJoCo and dm-robotics for simulating object rearrangement tasks. I am using these tasks to evaluate different approaches to learning policies for robot perception and control. I'm looking for some quick guidance on the AgentFlow SubTask APIs (I will also document my learnings here for other users to leverage).

Here is an explanation of my question:

I wish to create a task environment that operates over abstract actions (e.g. pick/place object x; a good example of such abstract actions is in transporter_networks which we intend to use as a baseline). I have been reading the APIs and understanding the schematic overview of AgentFlow:

The base task setup is very clear to me, however, the action space definitions and timestep preprocessors are less clear. From my current understanding action spaces must operate over the same timestep as the effectors in the base task. As a result, the subtask APIs can project different action specs but don't deal with action specs for actions that operate over different timescales to that expected by the basetask effectors (e.g. reconciling a pick/place action that operates over seconds with commanding actuator torques at 1000hz). To define abstract and temporally extended actions my understanding so far is that I need to consider the subtaskoption API as this enables me to define a policy and stepping procedure. I plan on exploring this route next but any clarification on whether this is the correct approach is appreciated. It is still unclear to me how I create a dm_env style environment with subtaskoptions as from my current understanding I can only compose graphs from these.

I have my own task object that leverages a subtask instance. It may be straightforward to build what I want with this, however, if possible I'd like to understand if I can use subtasks instances from AgentFlow to encapsulate tasks that leverage temporally extended actions.

Any advice is greatly appreciated, I am hoping that advice can fast track progress/inform decision making. I wish to use MoMa as it has the option to integrate with real robot setups. I will be sure to document my learnings and use of MoMa for others in the MuJoCo ecosystem to leverage.

[Peter David Fagan]

Closing for now as I have went with another approach.