the conv layer before bn layer has bias term?

[Kai Li]

https://github.com/jay-mahadeokar/pynetbuilder/blob/master/models/imagenet/resnet_50/train.prototxt#L18

I hear that the conv layer before bn layer may set bias_term to false.

Thanks.

[Jay Mahadeokar]

@kli-nlpr . Thanks for noticing this. Yes the bias in conv layer before bn seems to be unnecessary, I as also described here. I am able to train the resnet models and get accuracy as described in readme with current setting, not sure if removing bias will improve results.

[Kai Li]

Thanks @jay-mahadeokar

By the way, https://github.com/jay-mahadeokar/pynetbuilder/tree/master/models/imagenet/resnet_50
Can you post the solver.protxt for these resnet train test prototxts?

[Jay Mahadeokar]

The solver parameters I used are very similar to the one described in original paper. Strategy is multistep and I found that at LR 0.01 and 0.001, 100k iters is enough since error plateaus. See the plot in readme.

display: 20
average_loss: 40
base_lr: 0.1
lr_policy: "multistep"
stepvalue: 150000
stepvalue: 250000
gamma: 0.1
max_iter: 350000
momentum: 0.9
weight_decay: 0.0001
snapshot: 2000
snapshot_prefix: "snapshot"
solver_mode: GPU

[Kai Li]

Thank you very much! @jay-mahadeokar

[huangh12]

@kli-nlpr The first conv in resnet50 seems to indeed have a bias term, though it's before bn. The others however have no bias term. Do you know why？

[huangh12]

Problem solved. The conv1 doesn't connect to a BN directly, while the others do. So the bias term is ok.
Sorry to interrupt.