Main content
Strong evidence for expectation adaptation during language understanding, not a replication failure. A reply to Harrington Stack, James, and Watson (2018)
Date created: 2018-12-24 07:02 PM | Last Updated: 2024-02-15 03:14 PM
Category: Project
Description: Recent input affects subsequent language processing. One explanation for this holds that comprehenders adapt their implicit linguistic expectations based on the input, so as to facilitate efficient processing. A number of studies have identified support for this hypothesis. Harrington Stack, James, and Watson (2018) report a failure to replicate one of these studies (Fine, Jaeger, Farmer, and Qian, 2013). We show that, to the contrary, the data from Harrington Stack and colleagues constitute strong support for the hypothesis of expectation adaptation. Several factors contribute to the difference in conclusions. For example, Harrington Stack and colleagues argue based on differences in p-values, which is known to be problematic. We instead employ well-formed Bayesian measures of evidentiary support to assess replication success. Most critically though, the new experiments by Harrington Stack and colleagues differ in design from the experiments they aim to replicate. We correct for these differences by means of the same single-parameter belief-updating model previously employed by Fine and colleagues. The model provides trial-level predictions for the surprisal that comprehenders experience, based on previous input within and outside of the experiment. Trial-level analyses find that surprisal based on adapted expectations strongly predicts reading times in both the original and the replication data. In fact, once the differences in design are corrected for, the two data are highly similar; replication tests estimate the posterior probability of a replication success to be ≫ .9999. We show how the same belief-updating model also predicts trial-to-trial priming, cumulative priming, and the inverse preference effect in priming.
For now we have posted only the data sets we analyze, the model files, and the scripts for most figures and the Bayesian data analysis. The model we use is described in Appendix A of the manuscript.
All code, code explanations, and model results will uploaded prior to publication.
Files
Files can now be accessed and managed under the Files tab.
Citation
Recent Activity
Unable to retrieve logs at this time. Please refresh the page or contact support@osf.io if the problem persists.