Giters

huawei-noah / Pretrained-Language-Model

Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

About Prediction Distill

Fpw19980724 opened this issue · comments

commented

Hi, when I am doing prediciton distill task, the loss of the test set is pretty high and never decrease and the accuracy is 0.004, could you help me to solve this problem? The first intermediate distill task have worked, the loss is below 0.9.