alexa / bort

Repository for the paper "Optimal Subarchitecture Extraction for BERT"

alexa/bort Issues

The exact English pretraining data and Chinese pretraining data that are exact same to the BERT paper's pretraining data.
Updated 3 years ago1
Pre-training-Using-Knowledge-Distillation is better than Pre-training-Only for downstream tasks?
Closed 3 years ago2
Mask-Filling with pretrained BORT
Updated 3 years ago3
how to train model on another language?
Closed 3 years ago1
Huggingface support
Closed 4 years ago2
Create pretraining data with multiprocessing not Implemented
Closed 4 years ago1
bort pretrain
Closed 4 years ago5
I couldn't understand the configuration of the model. please can someone clarify?
Closed 4 years ago1
Can't download model.
Closed 4 years ago1
Can't download model again!
Closed 4 years ago1
Accuracy during fine-tuning is very low (only 0.68）
Closed 4 years ago1