This repository provides a PyTorch version of the pre-trained weights of BioBERT, a language representation model for the biomedical domain, proposed in (Lee et al. 2019).
Please go to the release section to download the models. The original TensorFlow models can be found here.
The DMIS GitHub repository for BioBERT provides details about BioBERT.
You should be able to run the models using the code from Hugging Face. I used version 2.2.1 and only adapted the code as follows:
global_step = 0
def evaluate
# Evaluate with the official SQuAD script # evaluate_options = EVAL_OPTS(data_file=args.predict_file, # pred_file=output_prediction_file, # na_prob_file=output_null_log_odds_file) # results = evaluate_on_squad(evaluate_options) # return results
def main
# Evaluate # result = evaluate(args, model, tokenizer, prefix=global_step)
# result = dict((k + ('_{}'.format(global_step) if global_step else ''), v) for k, v in result.items()) # results.update(result)
logger.info("FINISHED!")#"Results: {}.format(results))
#return results
def init (in BertForQuestionAnswering)
# name needs to correspond to variable in pretrained tf model to get weights correctly initialized self.classifier = nn.Linear(config.hidden_size, config.num_labels) self.activation = nn.Sigmoid()
def forward
logits = self.classifier(sequence_output)
Please see the LICENSE file for details. Downloading the weights indicates your acceptance of our disclaimer.
For help or issues using the pre-trained weights, please submit a GitHub issue. We only used our pre-trained models for question answering so far, so we most likely cannot help with other tasks. For more general questions about BioBERT, please refer to the DMIS GitHub repository for BioBERT.