UCSB replication of Stanford’s wave four ‘Misreporting’ study John Protzko, Jonatan W. Schooler The central hypothesis of this study was that people will, when forewarned, choose to not answer questions in a survey if they know that doing so would cause them to answer more questions. They will instead, ‘strategically misreport’, in order to get the survey over and their payment processed. Participants were told they would be asked some questions about sports. They first answered the question: “How much do you enjoy watching live sporting events on television?” on an unnumbered 5-point scale with response options [A great deal/A lot/A moderate amount/A little/Not at all]. At this point, participants were randomly assigned to either the experimental or control group. Participants in the control group moved on to the critical DV right away, whereas participants in the experimental group were first forewarned: If you answer the next question by saying you have watched five or more live sporting events on television during the past 12 months, you will be asked 20 more questions about sports. If you say you have not watched five or more live sporting events on television during the past 12 months, you will not be asked any more questions about sports. It was believed this forewarning would cause in increase in ‘no’ responses to the next question (that all participants answered): “During the past 12 months, since [CURRENT DATE – 12 MONTHS], did you watch five or more live sporting events on television, or did you not do that? [response options: Did watch five or more/Did not watch five or more]. Consistent with the replication model, participants who met any of the following criteria were excluded from the analysis: The participant failed the attention check, failed the captcha, reported completing the questionnaire more than once did not submit a completed questionnaire, or took more than 1 hour to complete the study. This caused us to drop 261 participants from the final analyses. Participants were randomly assigned to take the study as part of the 1st or 2nd 750 participants. Consistent with our analysis plan, we analyzed the 1st 750 participants first, then the 2nd 750 participants, then aggregated them into the overall analysis. 1st 750 participants We first ran a replication model, which included conditioning on the following demographic variables: whether male or female, whether Hispanic, age (dummy coded as 18-24, 35-44, 45-54, 55-64, 65+), ethnicity (dummy coded as white, black, or other), education (dummy coded as not a high school graduate, having attended some college, having an associates degree, or having a bachelors degree or higher), income (dummy coded as less than $30,000, $30,000-$50,000, $50,000-$75,000, $75,000-$100,000, and $100,000+), region (coded as northeast, south, midwest, and west), and dummy codes for missing data in any demographic variable. This replication model showed no effect of the forewarning on rates of responding ‘no’ and thus strategically misreporting themselves. Participants were just as likely to say they had watched five or more sporting events on TV if they had been forewarned (M = .616, seΔ = .026, n = 300), conditioning on demographics, than if they had not been forewarned (M = .649, seΔ = .026,n = 299; b (576) = -.033, p > .38, d = -.069, 95%CI = -.23 to .091). We also pre-registered that we would run a simple average treatment effect with no conditioning on variables using a probit analysis with robust standard errors. This analysis returned the same results as the overall model (bprobit (597) = -.085, p > .41). 2nd 750 participants In the 2nd 750 participants, we also failed to replicate the finding that participants would strategically misreport more if they had been forewarned than if they had not. In fact, those who had been forewarned, in the context of the OLS replication model conditioning on demographic variables, were nominally (though not statistically) more likely to say they had watched five or more sporting events on tv if they had been warned (M = .62, seΔ = .026, n = 311) than if they had not been forewarned (M = .609, seΔ = .026, n = 329; b (616) = .011, p > .77; d = .022, 95%CI = .177 to -.133). These results were unchanged when using a probit regression with robust standard errors (bprobit (638) = .05, p > .62). Full 1500 participants Finally, we aggregated the data into one dataset for the purposes of replicating the effect of forewarning on strategic misreporting. The results of the manipulation, in the context of the OLS replication models conditioning on demographics, were not statistically different between the 1st and 2nd 750s (binteraction (1212) = -.041, p > .43). Participants were just as likely to say they had watched five or more sporting events in the past year if they had been forewarned (M = .617, seΔ = .018, n = 611) than if they had not been forewarned (M = .628, seΔ = .018,n = 628; b (1214) = -.011, p > .67; d = -.023, 95%CI = -.134 to .089). The results remained the same when analyzed using a probit model with robust standard errors (bprobit (1498) = -.055, p > .4).