Is custom Python/Javascript assertion available through the evaluate() node module?
anthonyivn2 opened this issue · comments
Related to the question related to PR #668. I would like to be able to define custom assertions in Python/Javascript and then use it with the evaluate() node module
Yes, both of these assertion types are supported via promptfoo.evaluate.
In the case of javascript asserts, you can supply a function directly as the assertion value. I notice an inconsistency in how functions are called between external js files/inline js functions vs functions directly provided via evaluate, which will be addressed in #674
Ok noted, thank you!