QA Validation for Holdback First Run Enrollment
jaredlockhart opened this issue · comments
As part of the new Long Term Holdback mechanism, we need to verify that the clients can successfully enroll in both an outer holdback as well as an inner experiment all within a single enrollment pass on a first run client on Firefox Desktop.
Here is an example of an outer holdback experiment:
Here is an example of an inner first run experiment:
Can you please replicate these experiments on stage and configure a client that would pass the first run targeting on the inner client, and verify
-
The client will enroll on its first run in a copy of the inner experiment that does not require any other enrollments by satisfying just the first run targeting
-
The client will enroll on its first run in both the outer experiment and the inner experiment where the inner experiment requires enrollment in the outer experiment within a single enrollment pass and emit enrollment telemetry that reports its enrollment in both outer and inner experiments
We expect that a client that satisfies 1) should also be able to satisfy 2) however our initial tests indicate that a client that satisfies 1) may not satisfy 2) so please verify whether this is the case.
Let me know if you need any other information, thank you 🙏
┆Issue is synchronized with this Jira Task