Improved code coverage and tests refactoring
lbartoletti opened this issue · comments
QGIS Enhancement: Improved code coverage and tests refactoring
Date 2021/03/22
Authors
- Loïc Bartoletti (@lbartoletti)
- Sébastien Peillet (@SebastienPeillet)
Contact loic dot bartoletti at oslandia dot com
maintainer @lbartoletti
Version QGIS 3.22 / 3.24
Summary
QGIS, and in particular its API, is well covered by unit tests. However, we noticed during the last developments that there is still blind spots in the test coverage of these classes, despite of the existing ones. Moreover, some may be redundant (and cost build and CI time), some are different depending on the environment.
Proposed Solution
Missing tests
Write the missing tests on critical part and fix the ones that could be wrong as it could be encountered in the past (i.e the Wkt parser). As a first step, we propose to focus on geometry classes and map tools.
Refactoring
We also propose a slight refactoring and in particular to split into several parts some files that are too large (I think about you in particular testqgsgeometry.cpp with your 18713 lines...).
Python tests
We have had, and still have, differences results between the C++ API and the python API. We would like to add the equivalent of c++ tests on the python binding.
CI/Coverage
There is currently code in cmake to generate code coverage, we will investigate the possibility of generating these reports to view and measure the evolution of code coverage.
Votes
(required)
Please add the Grant-2021 label, thanks!
👍
For information, I have created a dedicated repository for testing: https://github.com/lbartoletti/qgis_coverage/tree/coverage
For now, the result is from a separate build on one of my machines and pushed in this repository. We plan to switch the automation to a public CI.
These reports are built from the coverage of the geometry tests: ${MAKE} qgis_coverage -j${NB_PROC} CMD_ARGS="-R qgis_*geometry*"
There is two reports available:
@lbartoletti i don’t think your test run is complete — eg the box3d class is well covered by the tests in https://github.com/qgis/QGIS/blob/master/tests/src/python/test_qgsbox3d.py yet the report states there is no coverage
Ah indeed, the Rexeg you’ve used to control which tests are run is excluding all the python geometry tests, and the specialised tests for the rectangle and box classes.