***Corrigendum***
We discovered errors in the computation of the binomial tests reported in the paper such that the computations inflated the p-values from each test. We report each error here:
- The binomial test on p. 2108 reports that 30 out of 45 participants made accurate judgments more often than chance. The p-value should be .036 (not < .0001 as reported).
- The binomial test on p. 2109 reports that 36 out of 49 participants made accurate judgments more often than chance. The p-value should be .001 (not < .0001 as reported).
- The binomial test on p. 2110 reports that 28 out of 46 participants made accurate judgments more often than chance. The p-value should be .184 (not < .0001 as reported). This analysis reflects a qualitative change in the conclusions drawn from Experiment 3 in the paper: ***participants' pooled accuracies were not significantly different than chance.*** (Their accuracies as a function of the type of problem remain statistically reliable, however.)
We regret any inconvenience as a result of these errors. The corrected analyses are reflected in the analysis scripts (the .R files) for each experiment in this OSF page.
***What is "DR1", etc?***
The names of the folders below (e.g., "Experiment 1") correspond to the descriptions in Kelly, Khemlani, and Johnson-Laird (under review). The code, data, and analysis scripts also reflect a separate abbreviation system used for tracking experiments. Hence, "Experiment 3" corresponds to "DR5", i.e., the **5**th experiment conducted for studying **D**urational **R**easoning.
----------
***How do you run an experiment from the code provided?***
The experiments are written in Node.js using the "nodus-ponens" package. To run an experiment on your local machine, install Node.js and then follow these steps:
Download and unzip the corresponding experiment code, e.g., "DR5-Code.zip".
Use your command line interface to navigate to the corresponding directory where the code is stored, e.g.,
$ cd ~/Desktop/Code/
Launch the experiment as follows:
$ node main.js
Point your browser to the "hostname" provided on the screen, e.g., "http://localhost:55152"
----------
***Where is the registration for Experiment 1 (DR1)?***
It is available in the linked project entitled:
"The consistency of durative relations".
----------
***What are the differences between the pre-registered analyses and those reported in the provided scripts?***
DR1 & DR2
- Used the inverse of latency instead of log of latency to determine outliers.
The log to inverse transformation change was to better capture fast outliers that were unlikely to reflect considered reasoning.
Whelan, R. (2008). Effective analysis of reaction time data. *The Psychological Record*, 475–482.
DR5
- We pre-registered 2 GLMMs - one for estimates of the main effects and interaction and one for an estimate of the predicted simple effect. Instead, we ran the main effects GLMM and used the new R package "emmeans" to get the simple effect estimate out of the one model.
The GLMM change was motivated by the superiority of taking all estimates from the same underlying model. The 2 models was an easy way to get estimates for all of the effects but would introduce a bit of extra variability from the separate model computations. Emmeans also automatically adjusts the significance tests for multiple comparisons.