A brief summery on the concessions we have to make in order to work with jsPsych.
In the lab we have very good control of the stimuli that we present using ZEP-x.y. Zep makes it easy to assign a participant to a given group manually using the control window. ZEP stores the data in its database and the user is able to extract the data to a ".csv" file on the PC on which can be imported in a spreatsheet program.
However, with jsPsych we lose some of the control we used to have in the lab.
Here I'm differentiating between two categories of concessions we have to make. The ones that are fixable to some extent, we can code something to work around problems that are more trivially solved in the lab. In contrast there are also problems that are really hard which I think are unsolvable.
- Currently we pick groups/lists randomly. Even with the server online it will be more difficult and less controlled than assigning a participant manually to a group.
- It's harder to import .csv files (but this makes it easier in the end)
- eg store stimulus parameters in javascript scripts.
- Storage and retrieval of data on ILS server is not as straightforward.
- Lack of control on the physical properties of the stimulus.
- We have no control over how loud the participant sets her audio.
- We have no prior knowledge on the display resolution and/or size nor the brightness of the display.
- A sentence might advance to the next line for participant 1, but not for participant 2.
- etc.
- German vs English keyboard layout, a response button might appear on a different location for another keyboard layout.
- Keyboard vs touchpad (although it might seem easy to ignore mobile devices, many laptops have touchscreens too). (It would be interesting to see whether there is a reaction time main effect of mobile devices versus PCs.)
- It is harder to see whether the participant is making a cup of tea during the experiment.
- Probably few participants have specialized equipment like EEG or an Eyetracker at home, even if the required jsPscych modules for controlling such an experiment are there.
We lose some control over an experiment by running it online instead of in the lab. Some of the concessions that we have to make are mitigated because its easier to measure more people online than participants in the lab. Measuring more people might reduce the noise that exists due to the lack of control. It depends on how much control you are willing to sacrifice in order to run an experiment online.