## Study Information ## **Research Questions** In this study, we will closely replicate an experiment reported by Eskine, Racinik, and Prinz (2011) . Eskine, Kacinik, and Prinz (2011) tested the hypothesis that moral judgments are influenced by our current emotional state. In their experiment, participants consumed a couple of swigs of a bitter, disgusting drink (Swedish Bitters), a sweet sugary drink (Minute Maid Berry Punch), or a neutral drink (water). Participants then evaluated the moral wrongness of a series of hypothetical scenarios. Results suggested that participants who drank the disgusting drink evaluated the behavior portrayed in the scenarios as significantly more morally wrong compared to participants who drank the sweet or neutral drinks. The authors further found some evidence to suggest that political orientation moderated this effect. There are three reasons why this work is a great candidate for replication, especially in the context of a laboratory class this semester at Tufts University, PSY 42, Emotion Laboratory. First, the main objective of my emotion laboratory class is to teach students how to do research that examines the influence of emotion on behavior; the topic of this study perfectly fits the bill. Second, this study is one of several candidates for replication featured as part of the Collaborative Replication & Education Project (CREP) . CREP uses crowdsourcing to encourage replication by undergraduate researchers. It is a study that has been approved by IRBs and conducted by undergraduate researchers at a number of academic institutions, including Illinois Institute of Technology, Southern Oregon University, Wesleyan University, University of Minnesota, and others. Third, a replication that draws on a larger sample size will enable us to estimate the size of the effect with greater precision. **Hypotheses** This replication will test the same hypotheses examined by Eskine and colleagues (2011). They reported their hypotheses on page 296 as follows: "If moral disgust really does stem from physical disgust, we hypothesized, then taste perception should affect moral processing such that a disgusting beverage should elicit greater moral disgust than a sweet beverage or a control beverage (water). We hypothesized that if conservatives are indeed more sensitive to disgust, then the taste manipulation should affect their moral processing more strongly than the moral processing of liberals." **Sampling Plan** We will register this study information document on the Open Science Framework prior to creation of data. **Data collection procedures** Data will be collected on Tufts campus by a set of up to 20 undergraduate experimenters. Sessions will primarily be scheduled to take place in the Emotion, Brain, & Behavior Laboratory in the Psychology Bldg. Others may take place in rooms reserved by experimenters in other Tufts campus locations. After providing informed consent, participants will consume one of three commercially-available beverages, Swedish Bitters, Minute Maid Berry Punch, or bottled water. Assignment to conditions will be random. They will consume two 1-teaspoon doses in a disposable cup, one dose prior to evaluating the moral wrongness of the first three hypothetical scenarios, described next, and the second before evaluating the last three. Participants will be asked to assess the moral wrongness of six hypothetical scenarios in one of two counterbalanced orders (see survey attachment). After making judgments of moral wrongness for all hypothetical scenarios, participants will rate some sentences for imageability, rate their impressions of the beverage they drank, provide demographic information, and describe what they think the study is about. To guard against participant expectancy effects, we will use deception as used in the original experiment. Participants will be told that the purpose of this study is to determine whether the motor movements involved with drinking influence judgments while reading about others. To enhance believability, Participants will be asked to use their dominant hand and to make their motions as fluid as possible when consuming the beverage, as if they're taking a shot. (Experimenters will demonstrate this action.) Participants will also be asked to place their cup in a specific marked location. In reality, the purpose is to determine whether the type of beverage influences moral judgments of wrongness. **Sample size** Up to 150 participants will be recruited for this study during the Spring 2017 semester. Participants will be recruited through online sources, including social media posts via researcher accounts on Twitter, Facebook, etc., TuftsLife.com, and the Psychology Department’s SONA credit and paid sites. In addition, flyers will be posted in the community, including on the Tufts campus and provided to participants to distribute to friends. Participants will be adults ages 18 years and up. Recruitment efforts may specifically target people in the college undergraduate age range as a reflection of the sample studied in the original experiment by Eskine and colleagues (2011). Participants will be asked to consider a list of beverage ingredients as part of the consent process; those who are allergic will be excluded from participation to prevent allergen exposure. **Sample size rationale** We will recruit as many participants as we can (up to N = 150) within the time available in the Spring 2017 semester. At a minimum, we seek to recruit N = 57 participants, as required by CREP. Our hope, though, is to recruit at least N = 75-150 participants. With N = 25 in each of three experimental conditions, we will have 80% statistical power to detect large effects (d = .81); with N = 50 in each condition, we will have 80% power to detect medium effects (d = .57). The effects were reported to be d = 1.09 and d = 1.22 by Eskine et al. (2011). **Stopping rule** Data will be collected until we reach a minimum of N = 57 participants. If we reach that target prior to the end of the Spring 2017 semester, we will continue collecting data until the semester ends. Variables --------- **Manipulated variables** We will manipulate beverage type on a between-subjects basis. The three levels of this categorical variable are 1) bitter, 2) neutral, and 3) sweet. **Measured variables** Participants will make judgments of moral wrongness on a visual analogue scale ranging from not at all wrong to moderately wrong to extremely wrong. The original authors measured line length using a visual analogue scale. Line length in our Qualtrics instantiation will be automatically determined based on participant positioning of a cursor along a scale with values set to range from 0 to 100 (the values will not be visible to participants). They will rate impressions of the beverage on enjoyment, bitterness, sweetness, neutrality, and disgust on a 7-point scale ranging from not at all to neutral to very much. Participants will rate how imageability of four sentences on a 7-point scale ranging from 1, difficult to imagine, to 4, neither easy nor difficult, to 7, easy to imagine. They will also be asked to provide demographic information including sex, age, year in college, major, ethnic background, and political orientation. Finally, they will be asked to describe what they think the study is about using an open-ended item. Please see Qualtrics survey in Materials component for specific wording of measured variables. **Indices** We will calculate one score for each participant that captures their moral judgments (average moral judgment across all hypothetical scenarios) and imageability (average imageability across all sentences). Design Plan ----------- **Study type** Experiment **Blinding** No blinding. Experimenters will be aware of which condition a participant is assigned to. Participants will be aware that they will consume a beverage and they will be aware of the ingredients in that beverage; they will be unaware of the condition label for that beverage (bitter, neutral, sweet). **Study design** The study uses a 2 X 3 between subjects factorial design. The two factors are political orientation, a measured variable (conservative, liberal), and beverage type, a manipulated variable (bitter, neutral, sweet). Our primary interest is in the effect of the experimental manipulation of beverage type on moral judgments. **Randomization** Participants will be randomly assigned to one of three beverage type conditions. Political orientation is a measured variable; participants will provide a label. Analysis Plan ------------- **Statistical models** Our confirmatory analysis plan is as follows: 1. As a manipulation check, we will examine whether beverage type influences impressions of the beverage (sweet, bitter, neutral, disgusting) by computing four GLMs with one factor, beverage type (bitter, neutral, sweet). There are four planned contrasts as follows: Sweet: bitter (-1) > neutral (-1) and sweet (2) Bitter: bitter (2) > neutral (-1) and sweet (-1) Neutral: bitter (-1) > neutral (2) and sweet (-1) Disgusting: bitter (2) > neutral (-1) and sweet (-1) The original authors did not report inferential statistics; they reported descriptive statistics that, by visual inspection of means, reflected these planned contrasts. 2. To test Hypothesis 1 that beverage type influences moral judgments, we will compute a GLM with one factor, beverage type (bitter, neutral, sweet). A planned contrast will assess whether bitter (2) > neutral (-1) and sweet (-1). The original authors did not report this specific contrast; they reported the bitter versus neutral, bitter versus sweet, and neutral versus sweet effects, which we will do as well. 3. To test Hypothesis 2 that political orientation moderates the effect of beverage type on moral judgments, we will compute a 2 X 3 between-subjects GLM with factors of political orientation (conservative, liberal) and beverage type (bitter, neutral, sweet). A planned contrast will assess whether conservatives exhibit bigger differences in the expected direction than liberals for the bitter (2) > neutral (-1) and sweet (-1) contrast: Conservative [bitter (2) > neutral (-1) and sweet (-1)] minus Liberal [bitter (2) > neutral (-1) and sweet (-1)] The original authors noted that the interaction between political orientation and beverage type was not significant and did not report the interaction statistics. 4. As an additional test of Hypothesis 2, we will compare a disgust group (bitter condition) to a nondisgust group (created by pooling participants from the neutral and sweet conditions). A planned contrast will assess whether conservatives exhibit bigger differences in the expected direction than liberals for the disgust > nondisgust contrast: Conservative [disgust (1) > nondisgust (-1)] minus Liberal [disgust (1) > nondisgust (-1)]. The original authors did not report this interaction contrast; they reported the disgust versus nondisgust effect within each of the two political orientation groups, which we will do as well. **Transformations** We do not currently plan to transform any variables. If inspection of variables indicates a substantial violation of analytic assumptions, we will report the results of the original proposed confirmatory analyses as well as the results based on a transformation or appropriate alternative analysis. **Follow-up analyses** For all factors in the GLMs described above, we will calculate partial-eta squared effect sizes. In addition to the contrasts noted as part of statistical models above, we will calculate a Cohen’s d effect size for each of the planned contrasts noted. We will also calculate Cohen’s d effect sizes for pairwise differences between beverage type conditions as well (bitter versus neutral, bitter versus sweet, neutral versus sweet). These values will be corrected for unequal sample sizes as needed (Hedge’s g). We will use a Z test to compare the replication effect sizes to the original effect sizes. If effects are potentially leveraged by outlying values, we will repeat the analysis without the outlying values. We will report results with and without such outlier exclusions. **Inference criteria** We will use null hypothesis significance testing with alpha = .05. We do not plan to adjust our alpha for multiple comparisons. A successful replication will be defined in these ways: 1) Finding a statistically significant effect of beverage type on moral judgments as described in the planned contrasts described in statistical models 2, 3, and 4 2) Finding replication effect sizes that are: a) not significantly different from the original effect sizes b) not statistically equivalent to 0 (lower and upper equivalence bounds will be set to whatever effect size we have 80% power to detect given the final sample size; with N = 50 per group, equivalence bounds will be d = +/- .57). **Data exclusion** The data collected from participants who do the following will be excluded from our analyses: a) correctly infer that the purpose of the study is to determine the effect of beverage type on moral judgments b) provide moral judgments for less than 3 of the 6 scenarios Analyses that examine political orientation will exclude participants who identify a political orientation other than conservative or liberal. If effects are leveraged by outlying values, we will repeat the analysis without the outlying values. We will report results with and without such outlier exclusions. **Missing data** GLM analyses will automatically exclude subjects missing one or more observations. If we have too many missing observations due to listwise deletion, we will run a parallel analysis in Mplus using full information maximum likelihood estimation with robust standard errors such that all available data will be included. **Exploratory analysis** None **Known differences between this replication study and the original study** 1. We will administer all manipulations and measures via a Qualtrics survey. Doing so will facilitate our ability to collect the data in a standardized way for every participant and will minimize the risk of errors and data loss given the number of experimenters who will be involved in collecting these data. For the key moral judgments measures, line length will be automatically determined by Qualtrics based on participant-positioned location along a scale with values set to range from 0 to 100. It is unlikely that this change will affect the key results because the moral judgments, imageability items, beverage impression ratings, and demographic items are all the same except that they’ll be shown on a computer screen instead of a piece of paper. 2. The original authors indicated that they counterbalanced the order of presentation of the six moral vignettes without specifying exactly how. In this replication, we will present two sets of three vignettes in counterbalanced blocks. Half the participants will judge one set after the first drink and the second set after the second drink; the other half of participants will experience the sets in the opposite order. The same three vignettes will always appear as a set, but we will randomize the order of those three vignettes within each set to minimize systematic differences between the vignettes in the amount of time that has passed since participants consumed a beverage at the start of the block. This approach may be different from the original authors' approach, but it is unlikely that this change will affect the key results; there is no theoretical reason to believe that a particular order is critical to produce the hypothesized effects. 3. Participants in the original experiment received compensation in the form of credit in partial fulfillment of course requirements. In this replication, participants will receive compensation in the form of money (USD$8). It is unlikely that this change will affect the key results; there is no theoretical reason to believe that the specific incentive to participate is critical to produce the hypothesized effects. 4. We have added procedures meant to enhance the cover story about motor movements being of interest. As noted above, participants will be asked to use their dominant hand and to make their motions as fluid as possible when consuming the beverage. They will be asked to place their cup in a specific marked location. We took these steps to enhance the cover story given the original authors’ suggestion that replicators try to sell it. The hope is that the enhanced cover story will reduce data loss due to participants correctly inferring that the purpose of the study is to determine the effect of beverage type on moral judgments. There is no theoretical reason to believe that improving the believability of the cover story is likely to affect the key results. 5. Our sample will be drawn from a different population of college undergraduates, Tufts University instead of Brooklyn College. These are both college settings thus the populations of interest are similar; nevertheless, it is possible that drawing from a different population may affect the key results.