Giters

armbues / SiLLM

SiLLM simplifies the process of training and running Large Language Models (LLMs) on Apple Silicon by leveraging the MLX framework.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[Feature request] Add ORPO finetuning

s-kostyaev opened this issue · comments

commented

Hi @armbues, thank you for this great project.

Please add ORPO finetuning to do SFT and DPO in one step https://arxiv.org/abs/2403.07691

commented

Great idea! I will add this to the roadmap.

commented

The official implementation of the "ORPO Trainer" can be found here.