We employed pre-trained BERT models (distillBERT, BioBert, and SciBert) for text-classifications of the titles and abstracts of clinical trials in medical psychology. The average score of AUC is 0.92. A stacked model was then built by featuring the probability predicted by distillBERT and keywords of search domains. The AUC improved to 0.96 with F1, precision, and recall increasing to 0.95, 0.94, and 0.96 respectively. Training sample size of 100 results in the most cost-effective performance.
