Training error: RuntimeError: tensor sizes does not match
secsilm opened this issue · comments
I clone this repo and train model using the following code:
python src/main.py train \
--use-bert \
--model-path-base models/nk_base9_base --bert-model "bert-base-uncased" \
--train-path data/02-21.10way.clean --dev-path data/22.auto.clean \
--learning-rate 0.00005 --num-layers 2 --batch-size 32 --eval-batch-size 16 --subbatch-max-tokens 500Then a RuntimeError occurred:
Not using CUDA!
Manual seed for pytorch: 1005542141
Hyperparameters:
attention_dropout 0.2
bert_do_lower_case True
bert_model 'bert-base-uncased'
bert_transliterate ''
char_lstm_input_dropout 0.2
clip_grad_norm 0.0
d_char_emb 32
d_ff 2048
d_kv 64
d_label_hidden 250
d_model 1024
d_tag_hidden 250
elmo_dropout 0.5
embedding_dropout 0.0
learning_rate 5e-05
learning_rate_warmup_steps 160
max_consecutive_decays 3
max_len_dev 0
max_len_train 0
morpho_emb_dropout 0.2
num_heads 8
num_layers 2
num_layers_position_only 0
partitioned True
predict_tags False
relu_dropout 0.1
residual_dropout 0.2
sentence_max_len 300
step_decay True
step_decay_factor 0.5
step_decay_patience 5
tag_emb_dropout 0.2
tag_loss_scale 5.0
timing_dropout 0.0
use_bert True
use_bert_only False
use_chars_lstm False
use_elmo False
use_tags False
use_words False
word_emb_dropout 0.4
Loading training trees from data/02-21.10way.clean...
Loaded 39,832 training examples.
Loading development trees from data/22.auto.clean...
Loaded 1,700 development examples.
Processing trees for training...
Constructing vocabularies...
Initializing model...
Initializing optimizer...
Training...
/opt/conda/conda-bld/pytorch_1579022034529/work/aten/src/ATen/native/LegacyDefinitions.cpp:70: UserWarning: masked_select received a mask with dtype torch.uint8, this behavior is now deprecated,please use a mask with dtype torch.bool instead.
/opt/conda/conda-bld/pytorch_1579022034529/work/aten/src/ATen/native/LegacyDefinitions.cpp:19: UserWarning: masked_fill_ received a mask with dtype torch.uint8, this behavior is now deprecated,please use a mask with dtype torch.bool instead.
/opt/conda/conda-bld/pytorch_1579022034529/work/aten/src/ATen/native/IndexingUtils.h:20: UserWarning: indexing with dtype torch.uint8 is now deprecated, please use a dtype torch.bool instead.
Traceback (most recent call last):
File "src/main.py", line 612, in <module>
main()
File "src/main.py", line 608, in main
args.callback(args)
File "src/main.py", line 564, in <lambda>
subparser.set_defaults(callback=lambda args: run_train(args, hparams))
File "src/main.py", line 312, in run_train
_, loss = parser.parse_batch(subbatch_sentences, subbatch_trees)
File "/home/ubuntu/projects/self-attentive-parser/src/parse_nk.py", line 1029, in parse_batch
annotations, _ = self.encoder(emb_idxs, batch_idxs, extra_content_annotations=extra_content_annotations)
File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in __call__
result = self.forward(*input, **kwargs)
File "/home/ubuntu/projects/self-attentive-parser/src/parse_nk.py", line 617, in forward
res, current_attns = attn(res, batch_idxs)
File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in __call__
result = self.forward(*input, **kwargs)
File "/home/ubuntu/projects/self-attentive-parser/src/parse_nk.py", line 358, in forward
outputs = self.residual_dropout(outputs, batch_idxs)
File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in __call__
result = self.forward(*input, **kwargs)
File "/home/ubuntu/projects/self-attentive-parser/src/parse_nk.py", line 135, in forward
return FeatureDropoutFunction.apply(input, batch_idxs, self.p, self.training, self.inplace)
File "/home/ubuntu/projects/self-attentive-parser/src/parse_nk.py", line 109, in forward
output.mul_(ctx.noise)
RuntimeError: The size of tensor a (486) must match the size of tensor b (290) at non-singleton dimension 0