Preprocessing data

Tidied data in EmpiricalData folder.

Spanton & Berry (2020) and Spanton & Berry (2021/2022/under review) pulled from OSF and preprocessed/organized given code and matlab scripts there.

MWW (2007) data made available to David Kellen, and preprocessed as in preprocess_mickes.R file

Resulting data:

SB2020_e1
SB2020_e2
SB2020_e3
SB2021_e1
SB2021_e2
SB2021_e3
MWW2007_e1
MWW2007_e2
MWW2007_e3

In some instances data and fits relating to SB2021_e1, SB2021_e2, SB2021_e3 are referred to as SB2022_e1, SB2022_e2, SB2022_e3 - they are the same data.

Models

Unequal-variance Gaussian signal detection model (in code: GaussianUVSDT (simple), baselineUVSDT (multiple)) Equal-variance Gumbel signal detection model (in code: GumbelEVSDT) Ex-Gaussian/Gaussian signal detection model (in code: ExGaussNormEVSDT)

Illustration of models

GraphsIllustration.R

Fitting models

Fundamentally, two procedures:

Data sets with 1 set of old/new items (per participant/condition): FitSimple.R (MWW2007 E1 - E3, SB2020 E1 - E3, SB2021 E1 - E2)
Data set with multiple sets of 1 set of new items, multiple sets of old items (per participant): FitMultiple.R (SB2021 E3). Here data was recoded to treat only 1 set of new items as 'new'.

Flow

Toplevel routine in FitModel.R
AnalyseGsquared.R: Calculate G^2 for best fits
AnalyseFits_graphs.R: Identify best fits, model comparison by AIC, create visualizations of empirical fit

ShinyApp

under https://nklange.shinyapps.io/SDTParameterizations/ (code in ShinyApp folder) for exploration of individual-level best fits, ROC predictions, residuals and parameter estimates

Simulations

SimulateFromModels.R: For analysis of confidence ratings, correlations of parameter estimates and model mimicry, we use the same set of simulations. All simulations are based on parameter estimates of best fits to data in SB2021 E1 and SB2021 E2.

For analysis of the confidence rating distributions, we generated data on a 24point scale by extending the parameter estimates for criteria from the original 6-point scale (so that 5 thresholds turned into 23 spread across the approximately same space, increments sampled by uniform and then scaled by original distance between thresholds.). Here we used the characteristics of the data sets in MWW(2007, E1) - 24 confidence rating points, 300 items per participant

For simulation for analysis of mimicry and correlation of parameter estimates, we simulated data using a large number of items and a 6-point confidence scale

Analyse distribution of confidence ratings vs sign/sigo

SimulateFromModels.R: Simulations based on fits to SB2020_e1 and SB2020_e2 CalculateRatingSDs.R: Relate standard deviations of confidence ratings and 1/sig_o in UVGaussian fits for data (MWW2007), and data generated by U-V Gausian, Gumbel, and Ex-Gaussian (and then fitted with U-V Gaussian)

Analyse model mimicry

PenalizeMimicry.R

Analyse effects of variance in item effects

SimulateItemVariance.R: Effects of manipulating item variance on ROC and parameter estimates (SB2022_e3)
SimulateItemVariance_v2.R: Effects of manipulating item variance on ROC and parameter estimates (SB2022_e1,SB2022_e2)

Analyse correlation of mu and sigma_o in U-V Gaussian fits

UVSDTcorrelations.R

singmann / altsdt