microsoft / BitNet

Official inference framework for 1-bit LLMs

Repository from Github https://github.commicrosoft/BitNetRepository from Github https://github.commicrosoft/BitNet

e2e_benchmark.py uses incorrect build path?

grctest opened this issue · comments

commented

I tried using the readme instructions for running the benchmark in Windows 11, however I got the following warning:

python .\utils\e2e_benchmark.py -m /models/Llama3-8B-1.58-100B-tokens/ggml-model-i2_s.gguf -n 200 -p 256 -t 4
ERROR:root:Benchmark binary not found, please build first.

The file does exist at BitNet\build\bin\Release\llama-bench.exe, however the e2e_benchmark.py file seems to reference an incorrect path:

bench_path = os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), "build/bin/llama-bench")

The run_inference.py file works as expected, and uses the following file path:

mainPath = path.join(buildDir, 'bin', 'Release', 'llama-cli.exe');

Thanks for reporting! Fixed in 80b94ae.