pytorch / rl

This issue is a list of contributions requests from the community.

How to use this list

If you're willing to contribute to the library, have a look at the list below and don't hesitate to pick up a task.
If you need guidance, refer to @vmoens for more information!
Once you pick up a task, assign the related issue to yourself, to make sure that no other collaborator is working on the same task at the same time (or create an issue if there isn't one already).

If you want to add an item to this list, start by raising an issue and mention that you think it would be appropriate to put it in the "call for contributions" stack.

If it's your first contribution, warm up with an issue marked with https://github.com/github/docs/labels/Good%20first%20issue label (and claim the issue so no one else does!)

New algorithms

New algorithms can be coded either in a free form or using torchrl's trainer class.
In any case, we would ask to the user to use hydra for the configuration, and to limit the number of extra dependencies as much as can be.

PILCO (no open issue yet)
Image augmentation is all you need #32
TQC algorithm (#1623)
A3C #1755

## New environment libraries

Safety gym compatibility
Procgen
Genesis

New modules and features

Reward-to-go #16
On-the-fly adaptation of alpha and beta in PRB #1575
Raise exception when sampling from empty replay buffer (#994)
Add an option to "squash" the observation dictionary in register_gym when there is only one observation (ie, not return a dict but a simple tensor)
Add a num_envs option in DMControlEnv to create a parallel env in just one call, e.g. DMControlEnv(name, task, num_envs=4) would run 4 parallel envs of the name, task dmc env.

Datasets

BugFixes

Colabs don't render in the doc
Raise an meaningful exception when one-hot specs are reshaped with a shape that doesn't match the last dim for all transforms that incur a change of shape #1904

Deprecation calls

Softly deprecate NormalParamWrapper in favor of NormalParamExtractor
Deprecate wrappers

rl/torchrl/modules/tensordict_module/exploration.py

Line 252 in 69d44f5

class AdditiveGaussianWrapper(TensorDictModuleWrapper):

and

rl/torchrl/modules/tensordict_module/exploration.py

Line 385 in 69d44f5

class OrnsteinUhlenbeckProcessWrapper(TensorDictModuleWrapper):

in favor of simple modules like

rl/torchrl/modules/tensordict_module/exploration.py

Line 31 in 69d44f5

class EGreedyModule(TensorDictModuleBase):

Solved issues

Thanks for contributing to TorchRL!

[DO NOT CLOSE] Call for contributions