Giters

symflower / eval-dev-quality

DevQualityEval: An evaluation benchmark 📈 and framework to compare and evolve the quality of code generation of LLMs.

Home Page:https://symflower.com/en/company/blog/2024/dev-quality-eval-v0.4.0-is-llama-3-better-than-gpt-4-for-generating-tests/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Test for pulling Ollama model is flaky

zimmski opened this issue · comments

commented

Not sure if directly related but when I try to run make test localy on our main branch the following test fails: cmd/eval-dev-quality/cmd TestEvaluateExecute/Model_filter/Ollama/Pulled_Model

Local version of ollama is 0.1.34 will update and try again

commented

Running make install-all did solve the problem for me locally. Will take a look at this Bug.