add header with provider information
pelikhan opened this issue · comments
We have a test scenario with multiple providers and a few tests. Would it be possible to add a header cell on the output column with the id/label of the provider?
Hi @pelikhan, I may not be understanding this ask correctly. Currently the id/label should appear at the top of the table in the header.
Ah, I don't see it.
Could this be because I use a custom provider?
description: summarize all files
prompts:
- summarize
providers:
- id: provider.mjs
label: gpt-4
config:
model: gpt-4
tests:
- vars:
files: src/rag/markdown.md
asserts:
- type: llm-rubric
value: is a summary
- type: factuality
value: Markdown is a text-based syntax to generate documents
- type: icontains
value: Markdown(accidental closing)
Ah, this is happening because the provider label is intentionally hidden when all providers are the same in the eval. Potentially could remove this behavior if people find it confusing or inconsistent https://github.com/promptfoo/promptfoo/blob/main/src/web/nextui/src/app/eval/ResultsTable.tsx#L756-L758
Great, that makes sense.
The provider is always the same JavaScript file, the interesting one is "label".
(Is label something I hallucinated?) It could de displayed on the UI of present always.
Just made #721 - I think we can just show the provider always, for consistency :) Thanks for flagging this
This change is included in release 0.56.0
We also need a way to display a label instead of the provider; since the provider id is always going to be the file name (provider.mjs in this case)
