Run External File from `tests.csv`

[Sangwoo Joh]

Thanks to #238 , tests.csv supports Python function calls for assertions.
However, __expected doesn't seem to provide the ability to run external files (not just Python). Assert can execute external files using the file:// prefix to value, which would be nice to have for __expected as well.

[Ian Webster]

Thanks for flagging this. #608 fixes handling for external assertion files that are not Javascript or Python.

Tested with the following:

promptfooconfig.yaml:

prompts:
  - "Write a tweet about {{topic}}"

providers:
  - openai:gpt-3.5-turbo-0613

tests: tests.csv

tests.csv:

topic,__expected1,__expected2,__expected3
bananas,javascript:file://assert.js,contains:file://contains_text.txt,contains:file://contains_json.json

assert.js:

module.exports = (output, { vars }) => {
  console.log(`Received ${output} using variables ${JSON.stringify(vars)}`);
  return {
    pass: true,
    score: 0.5,
    reason: 'Some custom reason',
  };
};

contains_text.txt:

banana

[Ian Webster]

Fix is released in 0.50.0. Thanks!

[Sangwoo Joh]

Wow, I didn't think it'd be fixed so quickly, thank you so much!!