A/B variations

[Marvin Frachet]

Is your feature request related to a problem? Please describe.

As a regular user, I want to be able to get insights of different variations of a flag so that I can provide the better experience to my user.

Describe the solution you'd like

In the dashboard, add the possibility to provide multiple variations to a given flag and provide ways to target specific users with each of the varations

Describe alternatives you've considered

Adding a UI in the insights pane to compare N different flags (still relying on boolean values) in charts, like drawing lines for the different flags. While it could work, it would imply creating the N different flags (with potentially not very meaningful names) and it may create confusion since nothing is very clear. In terms of experience, this looks more as a "quick and dirty" way to do the thing

Additional context

N/A

What has been done

• #12
• #14
• #15
• #20
• #24
• #23
• #22
• #21
• #25
• #26
• #27
• #29
• #30

[Marvin Frachet]

Make sure to sort the variants buckets when doing A/B tests.

For instance, if we decide to rollout 3 variants to 100% of the audience, assuming that the bucket range is between 0 and 10000, we can assume that:

• variant 1 should resolve when bucket is less than 3333
• variant 2 should resolve when bucket is less than 6666
• variant 3 should resolve when bucket is less than 10000

If we don't sort the variant buckets when resolving, a user+experiment+variant could be 5000 and receive the variant 3 since the predicate would yield "true".

progressively/packages/backend/src/ab/ab.service.ts

Line 178 in 90b591e

 return isInBucket(key, String(userId), rolloutPercentage);