huggingface / alignment-handbook

Robust recipes to align language models with human and AI preferences

Home Page:https://huggingface.co/HuggingFaceH4

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Method to disable evaluation

ZhiCheng0326 opened this issue · comments

Just sharing here. While attempting to disable evaluation, I find that setting only do_eval: False is unable to turn it off.
After studying the code, it appears that both do_eval: False and evaluation_strategy: "no" need to be set in order to fully disable evaluation.

Reference:
https://github.com/huggingface/transformers/blob/76fa17c1663a0efeca7208c20579833365584889/src/transformers/training_args.py#L1418