benchmark 测试的时候会卡住,如何解决呢?
2213601279 opened this issue · comments
张驰 commented
./benchmark -p /opt/Convert/flm/qwen-14b-chart-int4.flm -f ../example/benchmark/prompts/beijing.txt -b 1
Load (323 / 323)
Warmup...
finish.
AVX: ON
AVX2: ON
AARCH64: OFF
Neon FP16: OFF
Neon DOT: OFF
TylunasLi commented
卡住可能是qwen-14B-int4生成的结果停不下来, 可以考虑加入参数“-l 512” 限制输出长度为512 tokens。