This is the repo for our emnlp2020 paper: Pretrained Language Model Embryology: The Birth of ALBERT arxiv link: https://arxiv.org/abs/2010.02480 This repo is not completed. We will make all our code bases and pretrained model of Albert publicly available before the conference starts. Albert-base-v1 ckpt: https://drive.google.com/file/d/1OR2_weaRAeWdz4eY19l5EDDQ6Ao7Lg_Q/view?usp=sharing