Compare Zero-Epoch Prediction with Fine-Tuned Prediction as well as Validation Score Comparison

[meganjkurka]

🚀 Feature

In the Validation Prediction Insights, include the Zero-Epoch Response (i.e. how would the original foundation model response) as well as the BLEU (or other validation score) on the Zero-Epoch Response. This will show the user what the original foundation model would say vs their fine-tuned model as well as the validation performance improvement from fine-tuning.

Motivation

This will allow the user to quickly see the value of their fine-tuning exercise.

[Philipp Singer]

This can be already solved by running a separate experiment and set epochs=0, then one can compare the outputs in separate windows, or by downloading the csvs and comparing outside.
One idea could be to save the output for each validation run and then display as separate columns in the insights, but it is already pretty hard to navigate there.