guoday / CCF-BDCI-Sentiment-Analysis-Baseline

The code for CCF-BDCI-Sentiment-Analysis-Baseline

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

run_bert.py eval_loss计算错误

RingoTC opened this issue · comments

raw:
raw
changed:
changed
训练时,发现eval_loss的曲线太奇怪了,后来发现原版的pytorch_transformer把eval_loss的计算放在了
with torch.no_grad 里,于是修改代码为:

with torch.no_grad():
    tmp_eval_loss= model(input_ids=input_ids, token_type_ids=segment_ids, attention_mask=input_mask, labels=label_ids)
    logits = model(input_ids=input_ids, token_type_ids=segment_ids, attention_mask=input_mask)
    eval_loss += tmp_eval_loss.mean().item()

然后就正常了,不知道是不是这个原因?