marmelab / gremlins.js

Monkey testing library for web apps and Node.js

Home Page:https://marmelab.com/blog/2020/06/02/gremlins-2.html

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Interactions not completely deterministic despite seeding the randomizer

Piedone opened this issue · comments

We seed the randomizer as instructed in the docs as you can see in our code here. Despite this, we observed that the interactions are not completely deterministic, see here.

What is it that we can be missing? Thank you!

No idea. Did you add custom species / strategies ? Did you use Math.Random in them?

Yes, we do add a custom strategy here and use this list of species. However, all of this just configuring built-in gremlins.js functionality and we don't use any custom randomization logic there.

Do we understand correctly that the randomizer setup here should be enough for deterministic execution (if the given seen is constant, which it is)?

I'm sorry, as you're rendering JS through a templating system, I can't see the code that you're running. Could you build a self-sufficient repro in Codesandbox?

As it turned out, this wasn't an issue with gremlins.js. Rather, an animation on the site and its varying timing (due to the capabilities of the machine it was executed on) caused flakiness.

Thanks for your help!