[Exploration]: Percentage rollout with criteria

[Marvin Frachet]

This exploration is about putting in place a percentage base rollout with criteria.

What exists today

When a user requests Progressively, we assign them a unique identifier (generated with nanoid).

Each user is guaranteed to have the same consistent ID on subsequent requests because the generated nanoid is stored in cookies.

When a user requests with a previously generated ID to access a specific feature, we create a string that corresponds to the user ID concatenated with the feature ID. Something like

const concatenationOfUserIdAndFeatureId = userId + "-" + featureId

We can convert this string into a number using the murmur hash 3 algorithm which provides a 32-bit number. We know that murmur hash is well distributed and is good at pseudo-randomness.

Now, we have a number where the maximum value is the maximum 32-bit number. We can divide that number by something we can work with, like a number between 0 and 100.

const concatenationOfUserIdAndFeatureId = userId + "-" + featureId
const hashed = mumurhash3(concatenationOfUserIdAndFeatureId);
const ratio = hashed / MAX_INT_32;
const percentage = ratio * 100;

What we can do now, from the Progressively standpoint, is to say: “hey, I want to roll out the feature X to 20% of my audience”. With the previously computed percentage for a user/feature couple, it’s now possible to make the following comparison:

// the computation above is hidden for clarity
const percentage = ratio * 100;
const rolloutPercentageForAudience = 20 // meaning 20%

if (percentage <= rolloutPercentageForAudience) {
  // rollout the feature to the user
}

What problem do we have

In Progressively, it should be possible to rollout features to a percentage of the audience with criteria. For instance, it should be possible to roll out to people with an email containing “@gmail.com” etc.

But it should also be possible to rollout features to 20% of the people with an email containing “@gmail.com”

Since we don’t store users (and we don’t have a base to compare with), is it mathematically possible, to use the same technic we use with murmur hash to make it happen?

[Marvin Frachet]

As a temporary solution, we'll admit that people matching criteria will be part of the rollout, even if they are not in the percentage range