Training parameters used
evantancy opened this issue · comments
Hi there, I was just wondering what training parameters were used? Reading the comments from arguments.py, some default values in the argparse descriptions don't match the actual default values.
Thanks
We only used the default values in arguments.py, but you may get better performance after some parameter tuning.
Hi again, was the robot visible to humans during testing?
Also when varying the number of humans, was the model retrained for that specific number of humans and number of environment steps kept the same?
You can set the robot to be visible or invisible. By default, the visibility in testing is (and should be) the same as training.
The model is re-trained for a fixed number of humans because the gym environment is implemented with a fixed number of humans. But it is possible to set the observation space based on a maximum number of humans and simulate a crowd with a changing number of humans.
I see, I was trying to reproduce the results published in the paper, and was getting 84.4% success rate, collision rate 12.8% and timeout rate 2.8% for default training parameters, just changing sim.human_num = 10 in config.py. Could this due to the randomness of simulation environment, so the robot might need more training episodes to learn the optimal policy?
Is sim.group_human True or not? If yes, you can test other checkpoints, or change the random seed in arguments.py and retrain. If not, that sounds reasonable since our results in FoV environment were obtained with 5 people.
sim.group_human was set to False. I see. Just to confirm, are you referring to Fig. 5 in the paper? Also what was the FoV parameter for the robot when varying number of humans?
Yes.
In group environment, the FoV was kept to be 360 degrees. See the first two paragraphs in Section 4.A.(1) for details