Main content

Home

Menu

Loading wiki pages...

View
Wiki Version:
**Study Description** This study replicates and extends the findings from Study 1. In Study 1, we found that those who preregistered their predictions about how long it would take to solve an anagram task (which people reliability underestimate) updated their beliefs about their verbal ability more than those who did not preregister their predictions. Furthermore, preregistered predictions were much shorter than recalled predictions after attempting the anagrams (demonstrating the hindsight bias). Study 2 includes the conditions from Study 1, wherein participants either do or do not preregister their predictions about how long they will need to complete two anagrams. This study adds two conditions where participants do not actually attempt anagrams after preregistering or not. Instead, they are simply told that most people underestimate the amount of time needed to complete the anagrams. These added conditions test whether individuals need to experience an event that challenges their expectations for preregistration to facilitate belief change or if preregistration facilitates updating beliefs even in the absence of direct experience. **Sample** We planned to collect 1500 participants that passed our attention checks (described below) using an online sampling firm. We slightly exceeded this number. The data have already been collected at this time. The data have not been observed or analyzed other than to confirm that we had collected at least 1500 participants who passed our attention checks. **Exclusion Criteria** Participants were excluded from analysis if they failed to pass either of two attention checks or indicated that we should not use their data. Part way through the study, participants were asked this question: “I understand English. We are asking this question to identify people who may be responding randomly.” Strongly agree Agree Slightly agree Neither agree nor disagree Slightly disagree Disagree Strongly disagree On the demographics page (near the end of the study) participants were asked: “Are you currently using a computer? We are again asking this question to identify people who may be responding randomly.” Yes No Unsure Not applicable Need more information Finally, on the last page before the debriefing, participants were asked: “Researchers typically like to limit the data they analyze to only those participants who paid attention to the tasks and answered questions honestly. Researchers would prefer to exclude participants who did not pay close attention throughout or only tried to finish the study as quickly as they could. Below, we will ask you whether you think your data should be included in the data analysis, meaning that you believe you paid close enough attention to the study. You payment does not depend on how you respond. Regardless of your choice, you will receive full payment. Please answer honestly. Do you believe that your data should be included in analysis?” Yes, my data should be included in analysis No, my data should not be used in analysis We only included participants who answered “Agree” or “Strongly agree” on the first attention check, “Yes” on the second attention check, and “Yes, my data should be included in analysis” on the final question. All participants who did not pass all of these checks were excluded. **Analysis Plan** There are two outcomes of interest in this study. The main outcome is the extent to which individuals update their beliefs about their verbal ability. This will be calculated by subtracting participants' second verbal ability rating (assessed after the anagram task) from their first, such that more negative numbers mean more updating. The second outcome is participants' predictions about how long it will take them to complete the anagrams. Participants either preregister this prediction or are asked to recall it after the anagram task. We will analyze both outcomes using a 2 (preregistration vs. no preregistration) x 2 (complete anagrams vs. no anagrams) ANOVA. We are primarily interested in testing whether the effect of preregistration on each of the outcomes is moderated by experience with the anagrams (as indicated by an interaction in the ANOVAs). We will follow up these analyses by conducting independent samples t-tests for our two outcomes comparing the preregistration vs. no preregistration conditions within each level of direct experience. There are no covariates. **Additional Sample (May 1, 2018)** We collected the planned sample and conducted the planned analyses indicated above. These analyses failed to reveal the hypothesized replication of Study 1, that individuals rate their verbal ability lower after preregistering their prediction about their performance relative to those who do not preregister. The interaction between the preregistration manipulation and the manipulation of direct experience was also not reliable. It is possible that these tests, despite the relatively large sample, were underpowered since the direct replication portion of this study (the direct experience condition) had roughly half the participants of Study 1. To address this, we collected a second sample using the same exclusion criteria and target N as listed above. We plan to apply the same analysis plan to the combined sample. Given that this collection decision was made after observing the data, we will report p-augmented (Sagarin, Ambler, & Lee, 2014) alongside traditional p-values when reporting the results.
OSF does not support the use of Internet Explorer. For optimal performance, please switch to another browser.
Accept
This website relies on cookies to help provide a better user experience. By clicking Accept or continuing to use the site, you agree. For more information, see our Privacy Policy and information on cookie use.
Accept
×

Start managing your projects on the OSF today.

Free and easy to use, the Open Science Framework supports the entire research lifecycle: planning, execution, reporting, archiving, and discovery.