Assert `FAIL` due to `Error: spawn E2BIG`

[Sangwoo Joh]

It looks like the context and model output I used for my experiments exceeded ARG_MAX. I'm currently asserting using an external Python script, and I'm getting this error for some tests:

Error: spawn E2BIG
    at ChildProcess.spawn (node:internal/child_process:420:11)
    at spawn (node:child_process:757:9)
    at executive (node:child_process:348:17)
    at pythonScriptOutput (node_modules/promptfoo/dist/src/assertions.js:181:50)
    at new Promise (<anonymous>)
    at runAssertion (node_modules/promptfoo/dist/src/assertions.js:180:50)
    at node_modules/promptfoo/dist/src/assertions.js:82:30
    at node_modules/async/dist/async.js:151:38
    at replenish (node_modules/async/dist/async.js:448:21)
    at node_modules/async/dist/async.js:453:13

Is it possible to make the Python assert read a file instead of the command line argument?

[Ian Webster]

This is probably fixed by #638, which runs invokes python functions more elegantly instead of the current approach, which executes directly through the shell.

[Sangwoo Joh]

@typpo yes, I think that should fix this too, and maybe that fix will be released shortly?
Thanks as always.

[Ian Webster]

Aiming to release this tomorrow morning :)

[Ian Webster]

Change is released in 0.51.0

[Sangwoo Joh]

Many thanks!!!!