lmb-freiburg / Multimodal-Future-Prediction

Hi,

Thanks for your wonderful work! I am trying to replicate the training with the SSD dataset. Could I ask:

Which optimizer you used for the sampling and fitting neural network? And what are the learning rates you set for the optimizers?
Is it correct to use hyps, hyps_sigmas in the given test.py file to compute the loss for the sampling neural network? These two parameters are outputs from the make_graph function:
means, sigmas, mixture_weights, bounded_log_sigmas, hyps, hyps_sigmas, input_blob, output_blob, tmp = session.run(output, feed_dict={x_objects: objects, x_imgs: imgs})

Thank you in advance for your help!

Thanks for your interest in our work. Here are the answers:

We used Adam as optimizer. For the learning rate, you can find the details in our supplementary Figure 3.b (page 15) https://arxiv.org/pdf/1906.03631.pdf
Yes, you can do that. However, notice that the make_graph returns the output of both the sampling and fitting network. So if you want to train only the sampling network, you do not need to run the fitting network. So you can stop at:

Multimodal-Future-Prediction/net.py

Line 39 in d65d72f

out_hyps, out_log_sigmas = self.disassembling(output)

.

For training the networks, sampling and fitting, you can refer to the loss functions in net.py

Line 66 in d65d72f

    
           def make_sampling_loss(self, hyps, bounded_log_sigmas, gt, mode='epe', top_n=1):

Line 138 in d65d72f

def make_fitting_loss(self, means, bounded_log_sigmas, mixture_weights, gt):

Hope this helps,
Best,

Thanks for your super clear reply!

Feel free to re-open the issue if you have more questions.

Questions about the optimizers used for training the sampling and fitting neural networks