About activate_scale function

[Yang Qinhui]

Hello authors, thank you for your excellent work!
I would like to know if the function in the encoder module is the best combination for model training that you have found through continuous training and tuning?

def activate_scale(self, scale):
return - self.scale_activation(scale + 5) - 2
def activate_scale(self, scale):
return - self.scale_activation(scale + 5) - 2.5

[Florian Barthel]

Hi,
this activation function tries to solve several problems at once:

• It should clip the values by a upper bound (here 2 or 2.5) so that the GPU usage does not explode at some point during the training
• It should be differentiable
• It should be similar to a linear function, since the gaussian splatting renderer also applies an activation (exp) afterwards

We have found that this functions meets those criteria quite well. In the figure below, I have plotted this function. Depending on the available GPU memory, I have used different upper bounds. This way the training does not crash if the GPU memory does not suffice, due to large scales. Ideally, with unlimited GPU memory, this upper bound would be removed.

And the +5 just moves the function along the x axis. This helps to provide the newly initialized decoder to have values around -5. Activated by exp, this results in 0.007, which works quite well as a scale initialization.

[Yang Qinhui]

Hi, this activation function tries to solve several problems at once:

• It should clip the values by a upper bound (here 2 or 2.5) so that the GPU usage does not explode at some point during the training
• It should be differentiable
• It should be similar to a linear function, since the gaussian splatting renderer also applies an activation (exp) afterwards

We have found that this functions meets those criteria quite well. In the figure below, I have plotted this function. Depending on the available GPU memory, I have used different upper bounds. This way the training does not crash if the GPU memory does not suffice, due to large scales. Ideally, with unlimited GPU memory, this upper bound would be removed.

And the +5 just moves the function along the x axis. This helps to provide the newly initialized decoder to have values around -5. Activated by exp, this results in 0.007, which works quite well as a scale initialization.

Thank you！