Giters

huawei-noah / Pretrained-Language-Model

Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

For BinaryBERT, what is the value of config.dyna_hidden_size?

Phuoc-Hoan-Le opened this issue · comments

commented

For BinaryBERT, what is the value of config.dyna_hidden_size? Where is it initialized at exactly, what file?

commented

The paper take DynaBERT (Hou et al., 2020) sub-networks as backbones, so this is from DynaBERT configuration

commented

What is the size though? there are many different types of DynaBERT.