Main content
Time stand still: Effects of temporal window selection on eye tracking analysis
Date created: | Last Updated:
: DOI | ARK
Creating DOI. Please wait...
Category: Project
Description: The number of possible approaches to conducting and analyzing a research study—often referred to as researcher degrees of freedom—has been increasingly under scrutiny as a source of variability and challenge to the reproducibility of experimental results. Here we focus on the specific instance of time window selection for time series data. As an example we use data from a visual world eye tracking paradigm in which participants heard a word and were instructed to click on one of four pictures corresponding to the target (e.g., “Click on the hat”). We examined statistical models for a range of start times from 700–2200 ms following the beginning of the carrier phrase (with word onset at 1000 ms), and for each start time a range of window lengths from 300–1800 ms, in steps of 16.667 ms, resulting in 8281 unique time windows. For each time window we ran the same logistic linear mixed effects model, including effects of time, age, noise, and word frequency on an orthogonalized polynomial basis set. Comparing results across these time ranges shows substantial changes in both parameter estimates and p values, even within intuitively “reasonable” boundaries. In some cases varying the window selection in the range of 100–200 ms caused parameter estimates to change from positive to negative. Rather than rush to provide specific recommendations for time window selection (which differs across studies), we advocate for transparency regarding time window selection and awareness of the effects this choice may have on results. Preregistration and multiverse model exploration are two complementary strategies to help mitigate bias introduced by any particular time window choice.