BatchNorm sometime raise an error (ValueError: Expected more than 1 value per channel when training)

[Yoshiaki Yasumizu]

MEnet train can raise the error relating to BatchNorm. In this case, changing batch size in the yaml file can solve the error.

MEnet version :  0.0.3_220403
/home/yyasumizu/yyoshiaki-git/MEnet/MEnet/train.py:45: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://ms
g.pyyaml.org/load for full details.
  dict_input = yaml.load(f)
device :  cuda:0
dropped labels (<2 samples) :  Index([], dtype='object')
no processed pickle file. Now generating inputs...
/home/yyasumizu/anaconda3/envs/menet_dev/lib/python3.7/site-packages/numpy/lib/arraysetops.py:583: FutureWarning: elementwise comparison failed; returning scalar instead, but in the
 future will perform elementwise comparison
  mask |= (ar1 == a)
% Nan in the dataset : 27.16292205581145
[I 2022-04-18 00:41:00,169] A new study created in RDB with name: 220417_optuna_CV
/home/yyasumizu/anaconda3/envs/menet_dev/lib/python3.7/site-packages/optuna/distributions.py:563: UserWarning: The distribution is specified by [30, 3000] and step=100, but the rang
e is not divisible by `step`. It will be replaced by [30, 2930].
  low=low, old_high=old_high, high=high, step=step
[W 2022-04-18 00:42:13,102] Trial 0 failed because of the following error: ValueError('Expected more than 1 value per channel when training, got input size torch.Size([1, 330])')
Traceback (most recent call last):
  File "/home/yyasumizu/anaconda3/envs/menet_dev/lib/python3.7/site-packages/optuna/_optimize.py", line 216, in _run_trial
    value_or_values = func(trial)
  File "/home/yyasumizu/yyoshiaki-git/MEnet/MEnet/train.py", line 205, in objective
    y_pred = model(data)
  File "/home/yyasumizu/anaconda3/envs/menet_dev/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl                                          [28/71322]
    result = self.forward(*input, **kwargs)
  File "/home/yyasumizu/yyoshiaki-git/MEnet/MEnet/models.py", line 43, in forward
    x = self.input_layer(x)
  File "/home/yyasumizu/anaconda3/envs/menet_dev/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/yyasumizu/anaconda3/envs/menet_dev/lib/python3.7/site-packages/torch/nn/modules/container.py", line 117, in forward
    input = module(input)
  File "/home/yyasumizu/anaconda3/envs/menet_dev/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/yyasumizu/anaconda3/envs/menet_dev/lib/python3.7/site-packages/torch/nn/modules/batchnorm.py", line 136, in forward
    self.weight, self.bias, bn_training, exponential_average_factor, self.eps)
  File "/home/yyasumizu/anaconda3/envs/menet_dev/lib/python3.7/site-packages/torch/nn/functional.py", line 2054, in batch_norm
    _verify_batch_size(input.size())
  File "/home/yyasumizu/anaconda3/envs/menet_dev/lib/python3.7/site-packages/torch/nn/functional.py", line 2037, in _verify_batch_size
    raise ValueError('Expected more than 1 value per channel when training, got input size {}'.format(size))
ValueError: Expected more than 1 value per channel when training, got input size torch.Size([1, 330])
Traceback (most recent call last):
  File "/home/yyasumizu/anaconda3/envs/menet_dev/bin/MEnet", line 33, in <module>
    sys.exit(load_entry_point('MEnet', 'console_scripts', 'MEnet')())
  File "/home/yyasumizu/yyoshiaki-git/MEnet/MEnet/cli.py", line 87, in main
    args.handler(args)
  File "/home/yyasumizu/yyoshiaki-git/MEnet/MEnet/cli.py", line 17, in command_train
    train(args)
  File "/home/yyasumizu/yyoshiaki-git/MEnet/MEnet/train.py", line 272, in train
    study.optimize(objective, n_trials=N_TRIALS)
  File "/home/yyasumizu/anaconda3/envs/menet_dev/lib/python3.7/site-packages/optuna/study.py", line 410, in optimize
    show_progress_bar=show_progress_bar,
  File "/home/yyasumizu/anaconda3/envs/menet_dev/lib/python3.7/site-packages/optuna/_optimize.py", line 75, in _optimize
    progress_bar=progress_bar,
  File "/home/yyasumizu/anaconda3/envs/menet_dev/lib/python3.7/site-packages/optuna/_optimize.py", line 162, in _optimize_sequential
    trial = _run_trial(study, func, catch)
  File "/home/yyasumizu/anaconda3/envs/menet_dev/lib/python3.7/site-packages/optuna/_optimize.py", line 267, in _run_trial
    raise func_err
  File "/home/yyasumizu/anaconda3/envs/menet_dev/lib/python3.7/site-packages/optuna/_optimize.py", line 216, in _run_trial
    value_or_values = func(trial)
  File "/home/yyasumizu/yyoshiaki-git/MEnet/MEnet/train.py", line 205, in objective
    y_pred = model(data)
  File "/home/yyasumizu/anaconda3/envs/menet_dev/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/yyasumizu/yyoshiaki-git/MEnet/MEnet/models.py", line 43, in forward
    x = self.input_layer(x)
  File "/home/yyasumizu/anaconda3/envs/menet_dev/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/yyasumizu/anaconda3/envs/menet_dev/lib/python3.7/site-packages/torch/nn/modules/container.py", line 117, in forward
    input = module(input)
  File "/home/yyasumizu/anaconda3/envs/menet_dev/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/yyasumizu/anaconda3/envs/menet_dev/lib/python3.7/site-packages/torch/nn/modules/batchnorm.py", line 136, in forward
    self.weight, self.bias, bn_training, exponential_average_factor, self.eps)
  File "/home/yyasumizu/anaconda3/envs/menet_dev/lib/python3.7/site-packages/torch/nn/functional.py", line 2054, in batch_norm
    _verify_batch_size(input.size())
  File "/home/yyasumizu/anaconda3/envs/menet_dev/lib/python3.7/site-packages/torch/nn/functional.py", line 2037, in _verify_batch_size
    raise ValueError('Expected more than 1 value per channel when training, got input size {}'.format(size))
ValueError: Expected more than 1 value per channel when training, got input size torch.Size([1, 330])