`RangeError: Invalid string length`

[Sangwoo Joh]

I have a bunch of tests (4,500+) and Python bespoke asserts scripts.
When I run npx promptfoo eval -j16, it runs all jobs in parallel, but after they have finished, I got the following error:

/home/sangwoo-joh/type/work/promptfoo/node_modules/drizzle-orm/sqlite-core/columns/text.cjs:75
    return JSON.stringify(value);
                ^

RangeError: Invalid string length
    at JSON.stringify (<anonymous>)
    at SQLiteTextJson.mapToDriverValue (/home/sangwoo-joh/type/work/promptfoo/node_modules/drizzle-orm/sqlite-core/columns/text.cjs:75:17)
    at /home/sangwoo-joh/type/work/promptfoo/node_modules/drizzle-orm/sql/sql.cjs:157:73
    at Array.map (<anonymous>)
    at SQL.buildQueryFromSourceParams (/home/sangwoo-joh/type/work/promptfoo/node_modules/drizzle-orm/sql/sql.cjs:110:32)
    at /home/sangwoo-joh/type/work/promptfoo/node_modules/drizzle-orm/sql/sql.cjs:129:21
    at Array.map (<anonymous>)
    at SQL.buildQueryFromSourceParams (/home/sangwoo-joh/type/work/promptfoo/node_modules/drizzle-orm/sql/sql.cjs:110:32)
    at /home/sangwoo-joh/type/work/promptfoo/node_modules/drizzle-orm/sql/sql.cjs:132:21
    at Array.map (<anonymous>)

Node.js v19.9.0

• Ran on 16GB RAM, 16GB Swap, 8 Core Machine
• Node 19.9.0

[Sangwoo Joh]

Hmm... I reduced the number of tests to about 1k and it works fine, which seems like this error occurs when we have too many tests.

[Ian Webster]

Looks like we'd have to do something like this: https://stackoverflow.com/questions/29175877/json-stringify-throws-rangeerror-invalid-string-length-for-huge-objects