proposal: temporary suspension for changes w/o tests on vscode extension and language server
ssbarnea opened this issue · comments
As a desperate move for improving the test coverage on these two projects, I would like to propose a suspension of merging any changes that do not include tests. Reviews should continue to happen but if the change does not include testing that prevents regression, that review outcome should be "request changes, with link to this issue".
In general this is a normal requirement for a mature project but for these two, where we started to run CI testing very recently and where the (unmeasured) code coverage is likely below 5%, this rule would mean: you first need to help with one of more preparatory pull-requests related to testing before making other changes to the code.
I know that may delay introduction of desired bug-fixes but the reality is that if we have no tests, we have no proof that it works or that it will not regress with next change we make.
My hope that this will motivate everyone to spend some time on automating testing for these two projects. So far me and @webknjaz were the only ones that worked on this. We really need to bring these two project in-par with our other Python projects, where we do have a decent test coverage, usually above 90%.
Related: ansible/vscode-ansible#139
Alternatively, if there's no tests, there should be 3+ reviews from the team.
Alternatively, if there's no tests, there should be 3+ reviews from the team.
That makes sense. Anyway we know that rules/guidelines can be bypassed when really needed. That proposal is more about sending a message about code quality. In the end we should still use our common sense.