Sparse Reward Environments

Question

Sparse Reward Environments

bhairavmehta95 opened this issue 6 years ago · comments

Did you happen to see SAC's performance on sparse-reward environments?

I know the DIAYN paper trained on sparse rewards, but I was wondering if vanilla SAC (in your expts) had any luck solving things like Continuous MountainCar.

Tuomas Haarnoja · Answer 1 · Sat Mar 31 2018 07:23:45 GMT+0800 (China Standard Time)

We haven't tried spare-reward environments with the vanilla SAC. My intuition is that it will not work any better than other RL algorithms with Gaussian/Boltzmann exploration because of lack of temporal correlation in the exploration noise.

Bhairav Mehta · Answer 2 · Sat Mar 31 2018 23:25:20 GMT+0800 (China Standard Time)

Gotcha; that's what we seem to be seeing, but just wanted to make sure!

Ethan Brooks · Answer 3 · Mon May 07 2018 06:08:44 GMT+0800 (China Standard Time)

Could you clarify what you mean by temporal correlation in the exploration noise? Thanks.