jgamper / biobert-pytorch

Ready to use BioBert pytorch weights for HuggingFace pytorch BertModel :relieved:

Biobertology

Setup:

pip install biobert-pytorch==0.9 pytorch-transformers==1.2.0 torch==1.5.1

Ready to use BioBert pytorch weights for HuggingFace pytorch BertModel.

To load the model:

from biobertology import get_biobert, get_tokenizer

biobert = get_biobert(model_dir=None, download=True)
tokenizer = get_tokenizer()

Example of fine tuning biobert here.

How was it converted to pytorch?

Model weights have been downloaded from here and converted

by following the commands described here.pytorch

Specifically, using the command:

pytorch_transformers bert biobert_v1.1_pubmed/model.ckpt-1000000 biobert_v1.1_pubmed/bert_config.json biobert_v1.1_pubmed/pytorch_model.bin

followed by the renaming of the config file:

mv biobert_v1.1_pubmed/bert_config.json biobert_v1.1_pubmed/config.json

all of the other files inside the original weights file have been deleted.

Learn more about Bert & Transformer

1. The Illustrated Transformer

2. The Annotated Transformer

3. BERT Word Embeddings Tutorial