Giters

nivlab / nivturk

Tools for serving and storing data from online experiments.

Home Page:https://nivlab.github.io/nivturk/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[bug] interference across different nivturk experiments

szorowi1 opened this issue · comments

commented

Bug report: interference across different nivturk experiments

Users affected: @mbhare, @ines-as

Summary

Experimenters @mbhare and @ines-as ran into technical difficulties wherein a (partially) overlapping set of participants were automatically redirected to the completion page after starting an experiment. @ines-as was running a two-day longitudinal experiment (two separate NivTurk apps) and experienced the error on the second day of testing. @mbhare was running a cross-sectional experiment and experienced the error for participants that completed @ines-as' first day of testing. Importantly, participants were redirected with the completion code from the first session of @ines-as' experiment. The experiments were served on using different URLs, ruling out the possibility of a previous cache by URL.

commented

Resolution

Investigated by: @szorowi1

Summary

Users @mbhare and @ines-as were using the same secret key in their app.ini files. By sharing a secret key, the session cookies for the different NivTurk applications had the same signatures thereby creating interference across the applications.

Testing

@szorowi1 was able to identify, recreate, and solve the issue through the following tests:

Test 1

  • Make two new experiments by cloning NivTurk (prolific branch).
  • Configure the app.ini files so that both applications shared a secret key.
  • Complete the experiment for both applications, served on different URLs (ports), using the same Prolific ID.
  • Browser was not closed between experiment sessions (i.e. no cookie clearing)

Outcome: participant (@szorowi1) was automatically redirected to the completion page during the second experiment.

Test 2

  • Make two new experiments by cloning NivTurk (prolific branch).
  • Configure the app.ini files so that the applications had unique secret keys.
  • Complete the experiment for both applications, served on different URLs (ports), using the same Prolific ID.
  • Browser was not closed between experiment sessions (i.e. no cookie clearing)

Outcome: participant (@szorowi1) was able to complete both experiments without issue (i.e. recognized as two separate participants).

Test 3

  • Make two new experiments by cloning NivTurk (prolific branch).
  • Configure the app.ini files so that both applications shared a secret key.
  • Complete the experiment for both applications, served on different URLs (ports), using the same Prolific ID.
  • Browser was closed between experiment sessions (i.e. cookies cleared)

Outcome: participant (@szorowi1) was able to complete both experiments without issue (i.e. recognized as two separate participants).

Recommended fixes

  1. Use unique secret keys for every NivTurk application.
  2. Do not push secret keys to a Github repository to prevent accidental copying of keys.
  3. Update NivTurk docs to stress importance of using unique secret keys.