This repo contains information pertaining to the user study we present in the paper. This includes a datasets folder containing both the original and modified versions of the datasets we used in our evaluation, as well as the script that was used to generate the modified versions of the dataset. In the datasets folder, we've also included datasets that we considered but did not end up using in the study.
The study-planning folder contains both the full details of our study protocol EVM interview protocol.docx as well as legacy RMarkdown files used to brainstorm ideas other study designs that we considered.
There are also files in the root folder called:
appendix.md, where we include rationale about the implementaiton details of model checks that did not fit in the paper.interview-codebook.xlsx, which contains our qualitative coding spreadsheet documenting episodes of interest from our think-aloud and debriefing interviews evaluating EVM.model-check-parameterization.Rmd, which provides a walkthrough of a few alternative model sampling approaches we discuss in Section 6.1 of the paper.