almugabo / LMFit

experiments in finetuning language models

LMFit

experiments in finetuning language models

This "live" repository contains experimental work on finetuning language models (LM).

The focus is on decoder-only transformers. see: selected models : https://github.com/almugabo/LMFit/blob/main/selected_models.md

Its structure may/will change over time but for now it is organized around 4 topics

[1] Inference

with scripts on efficient generation of text from LM

[2] Fine Tuning

experimenting with fine tuning language models

[3] Evaluation

experimenting with the evaluation of language models

[4] others