Arbitrary annotations attached to test cases.

[Aman Patel]

Take a look at this: https://playwright.dev/docs/test-annotations#annotate-tests

What if all tests could document additional things like:

• type of test / Objects of the test
• Steps of the test (in english)
• Expected time of execution (for performance testing later on)
• Anything else...

We have 100s of tests for a medical software that is going to be used for medical studies. CFR Part 11 regulations require us to make sure that the software works as expected. The report must be in a human readable format. It would be nice if we could leverage the 100s of pest test we have and automatically generate additional compliance documentation about what the tests are testing.

Playwright seems to have this feature - it would be awesome to be able to support this via PHPUnit as well. If there is a workaround for this would love to know about it.

It would be possible to store the annotations externally, and then combine them later, but that would make maintaining them a lot harder.

[Sebastian Bergmann]

No new annotations will be added, only attributes. That being said, "arbitrary" and "anything else" does not sound very useful. Please open more specific issues, one for each specific attribute that you would like to see supported. Including an explanation of the use case you want to solve and the semantic you expect from this attribute.

[Aman Patel]

Thanks for quick response. A quick example would be something like this:

<?php declare(strict_types=1);
use PHPUnit\Framework\Attributes\DataProvider;
use PHPUnit\Framework\Attributes\TestDox;
use PHPUnit\Framework\TestCase;

final class ExampleTest extends TestCase
{
    #[TestDox('Patient Registration.')]
    #[TestAnnotation('expectedTime', '3s')]
    #[TestAnnotation('category', 'OQ - Operational Qualification')]
    #[TestAnnotation('objective', 'The objective of this test is to ensure that patient is registered correctly..')]
    #[TestAnnotation('steps', 'The steps are as follows: 1) Create a patient object with fake data (first name, last name etc), 2) Try to register the patient using our API, 3) Ensure that the API call succeeded, 4) Finally, ensure that the database was updated as expected with all the matching details generated in step 1.')]
    public function testPatientRegistersCorrectly()
    {
        $patient = Patient::factory()->make(); // create a fake patient record
        
        // call the registration API
        $result = apiCall($patient);

        // asert success response
        $result->assertSuccess();
        
        // assert the patient in the database
        $this->asertDatabaseHas('...');
    }

Having these annotations built in to every test would allow us to then generate compliance documentation such as this:

And we could also support a derived test for the elapsedTime for performance qualification (i.e. to ensure that the patient is registered in under <3s), since phpunit already collects timing information in the output.