Giters

tarpas / pytest-testmon

Selects tests affected by changed files. Executes the right tests first. Continuous test runner when used with pytest-watch.

Home Page:https://testmon.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Using `pytest --testmon --cov` to see merged coverage results

brendanator opened this issue · comments

commented

When I run pytest coverage along with testmon the coverage reports only the code that was run in the selected tests.

It would be incredible if the coverage reported the merged data from the tests that are selected and the rest of the data in .testmondata.

I'd love to get instantly updated test converage over a whole project. Is this possible?

commented

That is not unthinkable but quite difficult. Coverage data refers to the source code by line numbers, so it becomes invalid whenever you add or remove a line. I would say it's out of scope of this project for now.

commented

That is not unthinkable but quite difficult. Coverage data refers to the source code by line numbers, so it becomes invalid whenever you add or remove a line. I would say it's out of scope of this project for now.

Are there no other ways around this? I've just recently found your work @tarpas - so I am quite green when it comes to your code. But I've had a few hours of tinkering with it on my local machine.

What is exactly needed in order to make this work?

I've noticed that you erase the coverage reports for each test in /tests - why is that? Can't we just accumulate that json data?

EDIT:

wouldn't it just be an instance of caching and upserting changed or added "morfs"?

commented

Seems like commenting out this line

pytest-testmon/testmon/testmon_core.py

Line 346 in 67c8054

self.cov.erase()

doesn't seem to affect testmon, but allows coverage to keep track on the coverage data, and it looks like coverage is upserting their filenames: coveragepy/sqldata.py

meaning that it (should) accomplish what @brendanator wished for.

EDIT:

I did later on notice changes in testmon. So removing self.cov.erase()shouldn't be done. 😞

commented

As I wrote:

Coverage data refers to the source code by line numbers, so it becomes invalid whenever you add or remove a line.

Also to have complete picture: .testmondata is a "lossy" format. Only method checksums are stored.

Testmon would have to come up with "lossless" .testmondata format which is a challenge by itself (plus keep it performant and not eating too much space)

commented

This has been implemented and release to the extent it's technically practical/feasible. The results are not perfect, see discussion here: #86