About implement of AdaIN

[Nonu]

Thanks for providing source code.

I found that the implement of AdaIN (AdaptiveInstanceNorm2d) use constant value (zero and one) as running_mean and running_var, and they are unlike bias and weight which will be re-assign during forward.

So the input feature of this layer will not be normalize by it's statistic (mean and var), and just will be scale and shift by weight and bias calculated from style_tensor.

I wonder why not normalize the input features with statistics like instance_norm and then apply shift and scale? What effect will it have?

[Denis M Korzhenkov]

Hi! Please, notice that F.batch_norm at Line 39 is always evaluated in the training mode.
running_mean and running_var are just dummy variables.

[Nonu]

Oh, F.batch_norm will use statistics to normalize input in the training mode.

thanks for your reply :)