Conformance Validation
grampelberg opened this issue · comments
What problem are you trying to solve?
check does a great job of validating the health of the control plane and making sure that an install happened. Unfortunately, check is limited to only reading from the cluster and not running any actual workloads. This makes it challenging to know whether interactions between the control plane and data plane are working successfully.
How should the problem be solved?
Create a new tool that adds sample workloads to the cluster and runs something similar to integration tests against the installation. This will test things such as injection, policies and load limits for the cluster itself.
An example of conformance validation for kubernetes is: Sonobuoy.
Any alternatives you've considered?
It is possible to simply run through getting started today. This ends up exercising many of the same use cases. However, walking through getting started leaves much of the debugging and actual testing up to new users who might not know what they should be exercising. As an example, getting started does not currently test ingress, route based metrics or distributed tracing. All of these features could be exercised with a conformance testing suite.
How would users interact with this feature?
There should be a new set of CLI commands that can start, monitor and report on conformance testing.
This is somewhat related to #2079 .
Hi, would love to work on this issue as part of GSoC. I was wondering where in the codebase you envisage this being implemented? I've been familiarizing myself with it over the last few days and just want to try get some more context how I might tackle this.
The tests would be drawn from the integration test suite. The way you'd kick it off would probably be via a CLI command, such as linkerd conformance.
Perfect, thanks for the quick response, I'll focus on understanding the testutils and the current tests for now so.
Just in terms of testing the operation of the control plane, say for an operator who just installed linkerd. What aspects of operation would I look at testing first?
check is probably good enough for the operator who just installed linkerd. This is more about workloads, stressing the system and verifying that the environment is 100% sane. That's where the integration tests come in.
Ok, that makes sense, thanks. Definitely something I'd be interested on working on more. I've started a proposal for GSoC, planning on getting the draft in tomorrow and continuing to wrap my head around linkerd and this issue until the final submission. Either way, looking forward to getting a better understanding of it all!