**Analysis Plan** **Legend:** • IAT = Race Implicit Association Test • Pretest = taken on pretest. • Posttest = taken immediately after manipulation. • Followup = taken several days after manipulation. **Data preparation** Data preparation for the IAT will follow the “D2” scoring algorithm from Greenwald, Banaji, and Nosek (2003). The algorithm will delete trials with response latencies > 10,000 ms or < 400 ms, and keep error latencies in analyses. Participants will be excluded from all other analyses if IAT trials in any of the IATs they complete fulfill any of these four criteria: a. > 10% of trials <300ms overall in "combined" blocks (those used to generate the IAT score) across all IATs b. > 25% of trials <300ms in any one "combined" block c. > 30% error rate overall in "combined" blocks across all IATs d. > 50% error rate in any one "combined" block **Primary Analyses** There is some debate about the most effective method for analyzing a Solomon four-group design (Braver & Braver, 1988; Sawilowsy, Kelley, Blair, & Markman, 1994). Our analysis plan summarizes what we perceive to be the best available practice from the current literature. However, during the three data collections, we will continue to investigate and consult with experts in longitudinal design to see if this analysis plan can be improved. Any revisions will be recorded transparently along with the justification. For primary analyses, we will adopt an alpha = .01 as a means of reducing alpha inflation. We will first analyze the data to see if taking a pretest had an effect on IAT scores. We will conduct a 2 (Pretest Condition: Pretest-absent, Pretest-present) X 10 (Manipulation Condition) ANOVAs and see if there is a main effect of pretest condition and an interaction between pretest condition and manipulation condition on the posttest IAT. We will also conduct the same ANOVA but with the followup IAT as a DV instead of the posttest IAT. If there is a significant interaction between manipulation condition and pretest condition, we will conduct follow-up analyses to see if there are main effects of condition in the pretest-absent and pretest-present groups. To determine where group differences lie, we will also conduct eleven contrasts comparing posttest scores of manipulation conditions in the pretest-absent group with posttest scores of matching manipulation conditions in the pretest-present group. Our analysis plan for the pretest-present group follows recommendations from Rausch, Maxwell, and Kelley (2003). We begin by asking “Is there any evidence that groups differ in any way over time?” To do this, we will conduct a MANCOVA with the posttest and followup IATs predicted by condition, with the pretest IAT as a covariate. Then, we will conduct five follow-up ANCOVAS to better understand what specific group differences are occurring over time. These ANCOVAS will answer the following questions: 1. Do groups change differently from the pretest to the posttest? 2. Do groups change differently from the pretest to the followup? 3. Do groups change differently from the posttest to the follow-up? 4. Are the groups different on the average of the posttest and the followup? 5. Would the mean group change from posttest to follow-up be different had groups been equal on the outcome variable at the posttest? For questions 1 and 2, only the two out of four groups of the Solomon four-group design in which the pretest IAT was present will be analyzed. Two ANCOVAs will examine the effect of manipulation condition with the pretest IAT as a covariate. Posttest IAT scores will be the DV for the first question, and followup IAT scores will be the DV for the second question. For the other three questions we will conduct a primary analysis that includes all four groups of the Solomon four-group design. We will examine the main and interactive effects of manipulation condition and pretest condition (pretest-present versus pretest-absent). A secondary analysis will focus on the two of four groups in which the pretest IAT was present. This approach will afford high power to detect effects with the overall sample while still maintaining the ability to detect differences that account for variability in baseline IAT scores. In the primary analysis for question 3 and 4, we will conduct two ANOVAs examining the effect of manipulation condition, pretest condition, and the interaction between manipulation condition and pretest condition. The DV for question 3 will be a difference score between followup IAT and posttest IAT scores, and the DV for question 4 will be an aggregate score calculated by the mean of the posttest and followup IAT scores. For the secondary analysis, we will conduct two ANCOVAs examining the effect of manipulation condition on the difference score and aggregate score, respectively, covarying out pretest IAT scores. In the primary analysis for question 5, we will conduct an ANCOVA examining the effect of manipulation condition, pretest condition, and the interaction between manipulation condition and pretest condition on the difference between the followup IAT and posttest IAT scores with posttest IAT scores as a covariate. For the secondary analysis, we will conduct an ANCOVA examining the effect of manipulation condition on the difference score with pretest IAT scores and posttest IAT scores as covariates. Follow-up contrasts between conditions will be conducted alongside all of these analyses to determine where the group differences lie. As recommended by Braver and Braver (1988) we will compute meta-analytic effect sizes aggregating data from the pretest-absent and pretest-present conditions. Effect sizes will be computed by transforming the repeated-measure and independent-groups effect sizes into a common metric using techniques from Morris and Deshon (2003). **Secondary Analyses** All of these analyses are planned and will be reported either in primary text or in supplementary materials. Items A and B are secondary outcome measures. They are not the focus of the interventions, but there is practical and theoretical value in knowing if they do or do not follow the changes in implicit evaluation. Items C, D, and E are important tests of potential sample/setting circumstances. They are only of theoretical interest if they qualify the results of the primary analysis. If they do not qualify the primary results, then they are practically important in suggesting that the primary results are not a function of the tested samples and setting. Item F is a tertiary outcome measure. There is no strong theoretical basis for anticipating specific changes in implicit-explicit relations, but it is wise to discover if such effects occur as they would have theoretical implications if they were robust. Item G is also a tertiary outcome measure designed as a comparison check to prior results – not as a primary result for reporting. *A. Explicit attitudes.* 1. ANOVA examining the effect of condition on the posttest scores. 2. ANOVA examining the effect of condition on the followup scores. 3. ANOVA examining the effect of condition on the average of the posttest and followup scores 4. Follow-up contrasts between conditions will be conducted alongside each of these analyses to determine where the group differences lie. *B. Social judgment.* 1. Logistic regression examining the effect of condition on social judgment. 2. Follow-up contrasts between conditions will be conducted alongside each of these analyses to determine where the group differences lie. *C. Testing conditions (i.e., stimuli/attribute categories/background).* 1. Two ANOVAs examining the main effect of testing condition (Set 1 vs. Set 2) and interaction between testing condition and manipulation condition on posttest and followup scores. 2. Follow-up contrasts between manipulation conditions on the posttest and followup IATs will be conducted to determine where the group differences lie. *D. Data collection.* 1. ANOVA testing main effect of site on pretest, posttest, and followup IAT scores. 2. ANOVA testing interaction between site and condition on pretest, posttest, and followup IAT scores. *E. Attrition (Conducted for attrition within Session 1, within Session 2, and between Session 1 and Session 2).* 1. Logistic regression testing main effect of condition on attrition. 2. Logistic regression testing interaction between condition and age on attrition. 3. Logistic regression testing interaction between condition and religiosity on attrition. 4. Logistic regression testing interaction between condition and gender on attrition. 5. All analyses will be conducted with the overall sample and separately by data collection sites. *F. Implicit-explicit relations (Conducted separately for posttest data and followup data).* 1. Overall correlation between implicit and explicit preferences 2. General linear model with condition, implicit preferences, and the interaction of condition and implicit preferences in predicting explicit preferences. *G. Replicating the analysis strategy and results from Reducing Implicit Racial Preferences: I. A Comparative Investigation of 17 Interventions.* 1. Independent sample t-test contrasts between manipulation conditions and the control condition on the posttest IAT. **Deviations from Analysis Plan** We conducted the analyses as planned. However, we shifted our priorities in terms of what we reported. At the request of reviewers, we moved most of the Primary Analyses and C & D of the Secondary Analyses to the [Robustness Checks supplement][1]. As suggested by reviewers, we also removed reporting of Question 5's corresponding analysis in the main text and supplement, as it was prone to misinterpretation. We also added many analyses, some of which were self-initiated and some of which were spurred by reviewer suggestions. THe most dramatic change was the inclusion of data collection site as a covariate in many of our analyses. This change was done to be consistent with Study 2's analysis plan. Other analyses tapped into topics such as: - Aggregation of implicit preference data with Study 2 - Differential attrition analyses examining the role of pretest IAT scores - Differential attrition analyses related to IAT exclusions - Time between sessions as a moderator of implicit bias change - The role of data collection site on explicit preferences - The role of participant race on intervention effectiveness - Analyses using listwise deletion instead of pairwise deletion [1]: https://osf.io/fzmrh/