Giters

symflower / eval-dev-quality

DevQualityEval: An evaluation benchmark 📈 and framework to compare and evolve the quality of code generation of LLMs.

Home Page:https://symflower.com/en/company/blog/2024/dev-quality-eval-v0.4.0-is-llama-3-better-than-gpt-4-for-generating-tests/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Assess failing tests

zimmski opened this issue · comments

commented

Writing a test suite sometimes results in failing tests.

  • A failing test that compiles is always better than no test at all.
  • A failing test is worse than a passing test.
  • A failing test that covers nothing is worse than a failing test that covers something
  • ...