Automattic / wp-e2e-tests

Automated end-to-end tests for WordPress.com

Home Page:https://github.com/Automattic/wp-calypso

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Look at setting AB test groups on start page

alisterscott opened this issue · comments

@Stojdza says on #1488

@alisterscott What do you think about a solution to put back refresh in setABTestControlGroupsInLocalStorage(), but set it to false by default? And where it's needed to call refresh with true. Or otherwise - to be true by default, and where it's not needed or causing problems, set it to false.

I'm not sure is it applicable to all tests, but I think that way we would skip calling /themes page for every test.

@Stojdza: @bsessions85: I think we'll always want to set the AB test groups when we're visiting the start page and always refresh the page when we do which mimics the behaviour when using the AB test icon on for example wpcalypso.wordpress.com:

wpcalypso

I'm still not sure when we wouldn't not want to refresh since the AB tests can entirely change the start page, such as using a different flow which won't take effect until the test does something that moves pages

Reading the original reason for removing the refresh here: #1459 it seems that the refresh lost the query strings which is kind of a bug in itself since the user would also lose their place if they refreshed their browser on that page 🤔

But the solution we have works for the issue addressed in #1459 since we set the AB test before visiting start

Maybe we can just choose a better page to visit before start, like maybe /log-in is more lightweight?

I've made an attempt to clean things up a bit in #1491