Danish Legal Language Models Available Language Models for Danish This model is pre-trained on a combination of the Danish part of the MultiEURLEX (Chalkidis et al., 2021) dataset comprising 65k EU laws and two subsets (retsinformationdk, retspraksis) of the Danish Gigaword Corpus (Derczynski et al., 2021) comprising legal proceedings. It achieves the following results on the evaluation set.
Model Name
Loss
Accuracy
Maltehb/danish-bert-botxo22.3
7.038
coastalcph/danish-legal-lm-base84.8
0.651
coastalcph/danish-legal-bert-base80.1
0.878
coastalcph/danish-legal-bert-base82.5
0.768
coastalcph/danish-legal-xlm-base83.1
0.727
Model Name
EURLEX Val.
EURLEX Test
Maltehb/danish-bert-botxo73.7 / 42.8
67.6 / 38.2
coastalcph/danish-legal-lm-base75.1 / 46.5
69.1 / 41.9
coastalcph/danish-legal-bert-base75.0 / 50.4
68.9 / 44.3
coastalcph/danish-legal-xlm-baseTBA
TBA
coastalcph/danish-legal-longformer-base75.7 / 52.9
69.6 / 47.0
coastalcph/danish-legal-longformer-base + SD Penalty (Pezeshki et al., 2020 )76.1 / 52.9
69.9 / 47.0
The top-2 best models (coastalcph/danish-legal-longformer-basecoastalcph/danish-legal-longformer-base-sd
export PYTHONPATH=.
python src/mod_teacher_model.py --teacher_model_path coastalcph/danish-legal-lm-base --student_model_path coastalcph/danish-legal-lm-baseLongformerize pre-trained RoBERTa LM export PYTHONPATH=.
python src/longformerize_model.py --roberta_model_path coastalcph/danish-legal-lm-base --max_length 2048 --attention_window 128