rlgraph / rlgraph

RLgraph: Modular computation graphs for deep reinforcement learning

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[Core] graph_fn decorator needs an explicit `unsplit_ops` option.

sven1977 opened this issue · comments

So far, unsplitting is done implicitly, whenever split_ops=True. This is normally ok, but there are cases, where the output of the graph_fn has nothing to do in terms of nested structure with the input to the graph_fn. In these cases, it is important to e.g. switch splitting off and unsplitting on.

Example:
policy has a graph_fn into which we send the nn_input (e.g. a dict state with keys "a" and "b"). This function should then output actions (e.g. non-container simple actions). So the input (dict state space) does not match the output (flat actions) in terms of the nesting structure and the assumption to unsplit (the actions) in the same way as the split (of the state space) was performed will fail here.

I'm closing this issue as the problem can be easily solved by selectively flattening/splitting only certain input ops (unflattening of all returns is always automatic and required anyway).