Proposed Quality Workflow (QW) user guide - release 1
The Quality Workflow tool (QW) aims to help reduce the friction in creating and maintaining traceability matrices (later release could target hazards and risks).
Background and use-case
Within medical device software there are several design and development stages that should be tracked, versioned and a traceability matrix is a standard way of presenting this information. The Quality Management System used in WEISS is document based, which requires substantial manual effort in tracking linkages, updating of versions and ensuring that all approvals are current.
The QW allows users to track the different changes in design and development requirements, outputs, verification and validations. These different stages are linked together using github or gitlab as part of the normal development process. QW also will ensure that versioning of items are updated if their contents have changed, and notify when manual verification should be rerun due to changes in requirements or outputs.
Setup
Installation
The qw
tool requires python 3.9 or greater, if your project is in python then we suggest adding it to the developer requirements.
If not, you may want to create a virtual environment for your python, one option is the venv module.
For Unix:
python -m venv env --prompt qw # create virtual environment directory called `venv`
source venv/bin/activate # use the virtual environment
pip install git+https://github.com/UCL-ARC/qw.git#egg=qw # install qw
echo "venv" >> .gitignore. # ensure git ignores the `venv` directory
Configuration using Github
Initial setup and adding to repository
The qw
tool creates a .qw
directory that allows versions of each design and development stage to be tracked at release time.
Add this directory to your version control, so that there is a shared baseline at each release.
In the first step it also creates a set of GitHub actions and templates to ensure the tool can work correctly.
To allow the qw
to interact with GitHub you will need to create a fine-grained personal access token.
The final step of configuration then updates your GitHub repository with rules, and add templates for each stage of the quality workflow (and another
template if there is an issue or pull request that doesn't relate to the quality management documentation).
qw
Initialise At the top level of your git repository, run the intialisation step for github
qw init --repo github.com:username/reponame --service github
INFO: A ".qw" directory has been created
INFO: Github actions for qw have been added to your ".github" directory
INFO: Please commit the files after you have finished your setup
INFO: Rulesets and branch protection added for branches matching "main" to "stefpiatek/dummy-medical-software"
If you have a different development flow than pull requests into main/master then edit the qw
ruleset for the repository,
adding extra branches to the ruleset (e.g. develop
)
You can omit --service github
if the repo is at github.com
(as in
this case) and you can omit --repo <url>
if you have the desired
repository configured as the git remote named upstream
.
Setup Personal Access Token
-
Follow the GitHub instructions we suggest the following configuration options:
field value, user defined values defined by ${...}
Token name qw-${repository_name}
Expiration custom...
then choose 364 days from todayDescription QW tool for ${repository_name}
Repository access Only select repositories
-> Select the project repositoryPermissions Only add repository permissions, see next table repository permissions value Actions Read and write Administration Read and write Contents Read-only Issues Read and write Metadata Read-only Pull Requests Read and write Then click the
Generate token
button and copy the token (you can't get its value again if you close the page). -
Add the personal access token to your machine's keychain (uses keyring)
qw login
Please copy the personal access token and hit enter:
After you've done this, the system will check the personal access token. If you've configured for the remote URL https://github.com/stefpiatek/dummy-medical-software you would see:
INFO there are currently 27 issues and PRs
Can connect to remote repository 🎉
Setup for a new user, on an existing project that uses QW
The local and GitHub repository have already been configured to work with qw
, you'll just need
to add the personal access token
Customising configuration for identifiers
QW creates identifiers for User Needs and Requirements.
Design stage | format | example |
---|---|---|
User Need | URS-U${user need number} |
URS-U001 |
Requirement | REQ-${component short code}-${requirement number} |
REQ-SWR-0001 |
QW initialises with a System
component for requirements (used for generating a unique identifier, short code value is X
).
You can add extra components to the configuration by editing the /.qw/components.csv
name,short-code,description
System,X,Whole system requirements.
Drug dosage,D,Drug dosage calculation module.
qw configure
INFO: there are currently 27 issues and PRs
Can connect to remote repository 🎉
INFO: Writing templates to local repository, force=False
Local repository updated, please commit the changes made to your local repository.
Configuration using Gitlab
Intentionally left blank at the moment for brevity. Will aim for being able to implement this.
Using QW with github
QW uses existing issues and pull requests to track the different design and development stages along with managing risks and their mitigations.
Creating QW items
User needs
- In your github repository, Add a new issue, selecting the User needs template
- Fill out the required fields and any other information if it exists
- Hit
Submit new issue
and the issue will be rendered like this: - The first comment of this issue will be used by
qw
for tracking, but you can edit anything manually after theOther information
header and this will not interfere with the tool
Requirements
- In your github repository, Add a new issue, selecting the User needs template
- Fill out the required fields and any other information that may be useful
- The options for requirements type are:
- Functional
- System inputs and outputs
- Application programming interface
- Alarms and warnings
- Security
- User interface
- Data definition and database
- Installation and acceptance
- Operation methods and maintenance
- Networking
- User maintenance
- Regulatory
- The options for requirements type are:
- Hit
Submit new issue
and theissue will be rendered like this:- Note that the
design-component-d
has been added as "Drug dosage" was added as a component type with a short code ofD
.
- Note that the
- The first comment of this issue will be used by
qw
for tracking, but you can edit anything manually after theOther information
header and this will not interfere with the tool - The parent issue will have been updated with this issue number
Design outputs and design verification
- Create a pull request from github and follow the instructions on the template
- If the
qw-ignore
tag is added, then this PR does is not related to the medical device aspect of the software
- If the
- In this example, the pull request contains the design outputs and design verification. These can be added separately, where the design verification would also be linked to the requirement.
- QW will check that the chain of design items are able to be processed and fully signed off, when this is successful the github action will pass
- QW requires a pull request that is labelled as a
design-validation
ordesign-verification
to have at least one automated test that targets a specificdesign-output
,requirement
oruser-need
- Tests are tracked manually in
.qw/test_mapping.csv
where multiple issues are separated by;
, in the following structuretest name issue(s) targeted InputVerifierTests::testWeightIsNegative 40 InputVerifierTests::testConversionOfWeightUnits 4;23 acceptance_test_user_entry.docx 2 - The development team may want to write a script that allows them to get the names of all tests run and ensure that automated tests listed in the csv still exist, bonus points if you make this an automated test that fails if a test doesn't exist
- Adding the manual verification test scripts to version control is ideal, but you may also list the name
- the QW CI task will fail if:
- No parent issue is given for a PR
- A test doesn't exist which is mapped to the parent issue
- The structure of the
test_mapping.csv
has been altered, or;
has not been used to separate targeted issues - A test name is duplicated in
test_mapping.csv
- Tests are tracked manually in
Design validation
- QW does not require design validation documentation to be added to the repository,
but you may add the test script.
Ideally this would be in a plain text format. We suggest creating a
validation
directory and storing the validation information there - As with Design outputs and design verification, update the
.qw/test_mapping.csv
with the test script(s) that fulfill the user need. - Create a pull request from github and follow the instructions on the template
- A validation links to a user need, so link this type of issue in the PR
- Assign the pull request to the team member who will authorise the validation. Once they have validated it and signed any required paperwork, they should approve the PR.
Non-QW items
- If you'd like to create an issue that doesn't relate to the medical device then use the
Non QW item
issue template. This tags the issue withqw-ignore
so that QW does not parse the issue. - If you'd like to create a pull request that doesn't relate to the medical device then tag the pull request with
qw-ignore
.
Closing QW items when they are not resolved by a PR
-
There may be times when a QW-tracked issue is required to be closed not by a PR.
-
Then please add another section to QW information in the issue in the form:
### Closure reason #### type <!-- allowed types: duplicate, cannot replicate, not a defect, won't fix --> duplicate #### explanation Duplicate of #5
Versioning of QW items
The qw freeze
function is used to save the current state of the qw items in github, and to ensure that the versions of items are updated if their
information has changed.
This should be run regularly, which will update or add to the .qw
directory.
These changes should be committed and added to the main development branch by a pull request as a Non QW item.
qw freeze
An example incrementing a tag upon update:
Running WQ freeze Found 47 QW items
INFO: Requirement stefpiatek/dummy-medical-software#6 has been updated since last saved
Previous data:
- Description : Warfarin dosage should be calculated using based on patient age, gender and weight
- Parent User need: stefpiatek/dummy-medical-software#5
Current Data:
- Description : Warfarin dosage in mg/kg should be calculated using based on patient age, gender and weight
- Parent User need: stefpiatek/dummy-medical-software#5
Would you like to increment the version? (y/n): <prompt response from user - y>
INFO: Updated tag to "qw-v2"
INFO: There are 2 design outputs that link to this Requirement, please ensure one of them has the "qw-v2" tag if it resolves the updated information
...
INFO: :heavy_check_mark: 12 new QW items added to state
INFO: :heavy_check_mark: 47 QW items checked
INFO: Please commit the changes in the ".qw" directory
Example response when the change is trivial and does not warrant a change in the version of the QW item:
...
Would you like to increment the version? (y/n): <prompt response from user - n>
INFO: Tag kept at "qw-v1", data for the Requirement has been updated to the current state
...
Creating a documentation release
When you're ready to update the documentation in your Quality Management System (QMS), you can use the QW tool's release
command.
Running this will:
- Ensure that all issues and pull requests have been marked with
qw-ignore
or one of theqw-
item tags, raising an error (and stopping) to ensure all items are tagged - Ensure that all QW items have versions
- Ensure the entire chain
design validation -> design verification -> design output -> requirement -> user need
is consistent with QW rules, starting from the furthest stage of QW items. So if there is nodesign validation
, then the chain will start fromdesign verification
. If there was only auser need
andrequirement
s, then only these would be validated - Create word documents based on the QW template for export
- [name=Stef] Optionally? Create an html page that shows a burndown graph for each of the QW item types, showing the number completed and outstanding over time. Would this be useful?
qw release qms_docs
Creating a release for QW
Creating "qms_docs" directory if it doesn't already exist, and overwriting any previously exported files
INFO: :heavy_check_mark: 47 QW items checked
INFO: :heavy_check_mark: Documents have been written to the "qms_docs" directory
FAQ and common issues
- Can I import an existing project into QW
- There's nothing stopping this, though each issue and pull request would need to match the required format, and be tagged appropriately by the time you'd like to run a release. This may be a reasonable amount of work and we expect issues may need to be created.