Correlation of training speed influencing hyperparameters

[Lisa]

Hi there!

Here you have mentioned some parameters that can be tweaked to speed up training time:

@smandava98 thanks for the kind words! You can adjust the below hyperparameters correspondingly to optimization with fewer iterations:

• max_iter (max iterations, obviously)
• model.object.sdf.encoding.coarse2fine.step (increment one progressive level every N iterations)
• optim.sched.warm_up_end (learning rate warmup iterations)
• optim.sched.two_steps (decrease the learning rate at these iteration numbers)

Please also see #4 and the FAQ for details.

Let me present a quite extrem example. Below are two images of a training on the DTU data set 122. The first one is the visual wandb output after 5.000 training iterations, which is the end of the warm up phase with the suggested parameters. No c2f or lr decay has been applied so far. The other one is the same output after 5.000 iterations but with a warm_up_end = 100 and a c2f_step = 100. Still no lr decay but the training has unlocked all c2f levels.

Wandb Val Images

• max_iter = 5000
• model.object.sdf.encoding.coarse2fine.step = 5.000
• optim.sched.warm_up_end = 5.000

• max_iter = 5000
• model.object.sdf.encoding.coarse2fine.step = 100
• optim.sched.warm_up_end = 100

Clearly, the result is better when just doing "normal" training than running through all c2f levels. Especially the round starting shape is kinda ... preserved? I would be interested if you have some insights why that's the case and how the parameters depend on each other? Is there minimum (warmup) iteration amount for c2f to work properly?

Kind Regards
Lisa