boxed / mutmut

Mutation testing system

Home Page:https://mutmut.readthedocs.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

false positive string consistently proposed for `pytest.param`'s `id` parameter

emptierset opened this issue · comments

I am using mutmut v2.4.1.

As shown here, a common method for parameterizing tests in pytest is via pytest.mark.parametrize and pytest.param. The latter has a parameter id that can be used to assign an informational string to each test case. I daresay that no one would ever want to actually assert the content of these strings, so this has forced me to do something like this:

# mutmut_config.py

import os.path


def pre_mutation(context):
  _, filename = os.path.split(context.filename)
  is_test_file = filename.startswith("test_")

  line = context.current_source_line.strip()
  is_pytest_param_id_line = line.startswith("id=") and line.endswith(",")

  if is_test_file and is_pytest_param_id_line:
    context.skip = True

The problem is that this forces me to format every pytest.param on multiple lines, even if they are sufficiently simple and short to fit on a single line. Even if I wrote a more sophisticated regex to identify id=.* within a single line, I'm not aware of a way to prevent mutmut from mutating specifically the value for id, so I need to split it across lines anyway to mark the entire line containing only id with a "no mutate" pragma.

Is there a better way I could be doing this or a way to improve the experience?

Well there's a much deeper problem here. Mutation testing should NEVER mutate the tests themselves. So that's what you need to fix.

Ah, thanks. I guess I misunderstood the concept. For what it's worth, I did uncover a few bugs by running mutmut on my tests, but that makes sense in retrospect.