Wait between scenario runs

[Nik Skoufis]

It took me an embarrassingly long time to realise that

    config:
      runStrategy:
        minSecondsBetweenRuns: 600
        maxSecondsBetweenRuns: 900
        strategy: "random"
    scenarios:
    - name: Scenario A
    ...
    - name: Scenario B
    ...

causes PowerfulSeal to do the following:

1. Randomly select scenario A or B, and run it
2. Immediately select the other scenario and run it
3. Only now wait between 600 and 900 seconds

What I had assumed would happen was:

1. Randomly select scenario A or B, and run it
2. Wait 600 to 900 seconds
3. Select scenario A or B, and run it
4. repeat forever

Would it be useful to add a mode that does this?

[Mikolaj Pawlikowski]

Yes, I think that it would!

Better yet, you're not the first person to run into this, maybe let's just change the behaviour to the one you described. The least astonishment principle in action.

Thanks for the issue!

[Chris Stanaway]

I'd prefer this to be configurable. It could be difficult to manage the min/max times. In our case, we run large shared clusters and have teams create their own scenarios in their own namespaces (and use OPA Gatekeeper so scenarios can only target pods in the namespace in which the scenario was created (as well as other restrictions)). As the number of scenarios increase, the time between a particular scenario is executed increases significantly.

Some teams want to control how frequently a scenario is executed (e.g., randomly once a day). They can do that today since we set min=max and they can then use the probability setting. As scenarios are added by other teams, those existing scenarios will run less and less often (or somehow know when scenarios are created/deleted and everyone adjust their probabilities - not really a workable solution).