Date created: | Last Updated:

Category: Project

Description: Abstract Meta-d’/d’ has become the quasi-gold standard to quantify metacognitive efficiency because meta-d’/d’ is widely believed to control for discrimination performance, discrimination criteria, and confidence criteria even without the assumption of a specific generative model underlying confidence judgments. Using simulations, we demonstrate that meta-d’/d’ is not as model-free as previously thought: Only when we simulated data using a generative model of confidence according to which the evidence underlying confidence judgements is sampled independently from the evidence utilized in the choice process from a truncated Gaussian distribution, meta-d’/d’ was unaffected by discrimination performance, discrimination task criteria, and confidence criteria. According to five alternative generative models of confidence, there exist at least some combination of parameters where meta-d’/d’ is affected by discrimination performance, discrimination criteria and confidence criteria. A simulation using empirically fitted parameter sets showed that the magnitude of the correlation between meta-d’/d’ and discrimination performance, discrimination task criteria, and confidence criteria depends heavily on the generative model and the specific parameter and varies between negligibly small and very large. These simulations imply that a difference in meta-d’/d’ between conditions does not necessarily reflect a difference in metacognitive efficiency but might as well be caused by a difference in discrimination performance, discrimination task criterion, or confidence criteria. Flies This files provide all code necessary to replicate the results and Figures. All analyses were performed using version 4.0.3 running on Windows 10, using the R packages tidyverse 1.3.0, plyr 1.8.7, ggplot2 3.3.3, rjags 4.13, coda 0.19.4., R.utils 2.12.0, gridExtra 2.3, Rmisc 1.5.1, snow 0.4.0 and doSNOW 1.0.20. To replicate Simulation 1, download all files to your local computer and place all files in the same directory. Then, run the file "Mratio_byModels_analysis.R.R". The file Mratio_Plotting_v2023.R contains the code to plot the data. The file "Mratio_PlotCognitiveModelsAndMethods.RData" contains all the Figures. To replicate Simulation 2, run the file "Reanalyze_RouaultEtAl2018Exp2_ConfidenceModels.R". Depending on your hardware, this might take a while. The results are stored in the files "Correlations_RouaultEtAlExp2_ConfidenceModels.RData", "Simulations_RouaultEtAlExp2_ConfidenceModels.RData", and "ModelFits_RouaultEtAlExp2_ConfidenceModels.RData".