Testing
nibra opened this issue · comments
The idea is to automatically setup the test environements using docker-compose, controlled by a Robo task (see RoboFile.php). The setup should not only serve core development, but also persue extension developers to use the testing environment to improve the quality of their products.
Unit Testing
For unit tests, there's not necessarily needed a separate environment, but it can help anyway (see Isolating PHPUnit).
- Docker based test setup
- Improve the current unit testing coverage
- The overall coverage should at least be at 70%.
- Coverage of a single package should at least be at 80%.
Integration (Functional) Testing
For integration (functional) tests, we need an installed Joomla, i.e., initialised database, mailcatcher, and so on.
- Docker based test setup
System (Acceptance) Testing
For system (acceptance) tests, additionally the selenium grid is needed.
- Docker based test setup for CLI
- Docker based test setup for API
- Docker based test setup for Browser
- Page Objects, see #108
Security Testing
It makes sense to add a security test level using the OWASP Zed Attack Proxy and/or the Vega Vulnerability Scanner.
- Docker based test setup
We need extensions can also be tested.
With that, extensions on JED can have a status about unit test, security checked, etc.
Updated the issue description accordingly.