google-deepmind / reverb

Reverb is an efficient and easy-to-use data storage and transport system designed for machine learning research

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Uniform sampling without replacement until everything is sampled once

simon-bachhuber opened this issue · comments

I would like to create a Table object where every unique trajectory is sampled exactly once, until everything is sampled once, then it should reset. I think one might be able to achieve something like that with priorities, i.e.

key| priority
001| 1.0
002| 1.0
...
999| 1.0

Draw with reverb.selectors.Prioritized sample with key=002

key| priority
001| 1.0
002| 0.0
...
999| 1.0

Repeat 998 times

key| priority
001| 0.0
002| 0.0
...
999| 0.0

Reset all priorities to 1.0

key| priority
001| 1.0
002| 1.0
...
999| 1.0

Is that possible?
Thank you :)

Your idea would work, but:

  • after each sampling you would have to update priority of sampled item(s).
  • for performance you might want to sample multiple elements at once and then within a single batch elements might be duplicated.

Another option would be to have 2 tables, both with max sample count per element == 1. One table would contain your data initially, the other would be empty. Then you sample from table 1 and insert sampled data to table 2. Once table 1 is empty, "swap" the tables. This way you can batch-sample many elements for performant execution.

Closing this issue to keep the backlog clean. Please reopen if you have follow-up questions.