**Principal Investigator(s):** **Laura Stoker** University of California, Berkeley Email: stoker@berkeley.edu Home page: https://polisci.berkeley.edu/people/person/laura-stoker **Amy E. Lerman** University of California, Berkeley Email: alerman@berkeley.edu Home page: https://amylerman.weebly.com/ **Alexander Sahn** University of California, Santa Barbara Email: asahn@berkeley.edu Home page: https://www.alexandersahn.com/ **Sample size**: 3510 **Field period**: 06/25/2019-03/15/2020 **Abstract** Governments, interest groups, and the mass media routinely use quantitative information to depict the severity of problems facing the nation. When doing so, they can choose to report the (unacceptably low) percentage of people experiencing good outcomes—as in the percentage of high school students obtaining the degree—or the (unacceptably high) percentage of people experiencing bad outcomes—as in the percentage of high school students dropping out. Likewise, policy initiatives to improve matters can be framed as increasing the incidence of good outcomes or as decreasing the incidence of bad ones. Despite the ubiquity of these rhetorical choices in political discourse, there is no research into whether they matter to public opinion. Existing studies of negativity bias, loss aversion, and attribute framing do, however, support the expectation that they will matter. We carried out two sets of experiments to open up this line of inquiry. Each set consists of four experiments focusing, respectively, on drinking water quality, hospital deserts, high school achievement, and marine debris. The first set presents problem information, randomizing whether it is framed negatively or positively, and the second presents a policy solution, again with the frame randomized. All four experiments within each set showed clear and consistent results. When problems were negatively (vs. positively) framed, people expressed more anger and worry, and judged the problem more serious and more worthy of government attention. The public’s reaction to policy solutions, by contrast, were utterly unaffected by the frame. **Hypotheses** H1: Expressions of concern about the negatively-framed problem should exceed those found for the positively-framed problem—in terms of anger, worry, judgments of seriousness, and judgments regarding government priority. H2: Support for the loss-framed policy should be greater than support for the gain-framed policy—in terms of approval, willingness to sign a petition, and willingness to volunteer. H3: The effect of equivalence framing on both sets of dependent variables will be larger when more emotionally evocative language is used to describe the negative condition. In our study, this means we should find larger framing effects in the Marine Debris and Drinking Water Quality experiments than in the High School Achievement and Hospital Deserts experiments. **Experimental Manipulations** In this study, respondents were given either a problem or a policy question. The problems and policies were assigned randomly from a list of 4, which included: 1. Drinking Water Quality 2. Hospital Deserts 3. High School Achievement 4. Marine Debris For each problem, we randomly assigned respondents to one of two potential problem frames, either positive or negative. In the negative frame, respondents were told that the bad outcome was present at a rate of 20% while in the positive frame, respondents were told that the good outcome was present at a rate of 80; we only used problems where the 20%/80% breakdown was roughly accurate, and proved links to relevant studies in the debriefing materials. To illustrate, this is the wording Hospital Deserts treatments: > The chance of surviving a life-threatening medical emergency can depend on how long it takes to get to a hospital. According to a recent report, [20% / 80%] of Americans live [too far from / close enough to] a hospital to get adequate care in case of an emergency. The policy experiment treatments first introduced the problem and then described a policy initiative designed to ameliorate it. In the negative condition, the problem was described in terms of the prevalence of bad outcomes (20%) and the policy was portrayed as reducing that from 20% to 10%. In the positive condition, the problem was described in terms of the prevalence of good outcomes (80%) and the policy was portrayed as increasing that from 80% to 90%. To illustrate, this is the text of the Hospital Deserts conditions. > The chance of surviving a life-threatening medical emergency can depend on how long it takes to get to a hospital. According to a recent report, [20%/80%] of Americans live [too far from/close enough to] a hospital to get adequate care in case of an emergency. <br><br> Representatives in Congress have proposed a policy that would decrease the number of Americans that live [too far from / close enough to] a hospital to get adequate care. The policy proposal calls for a coordinated effort across the 50 states, the creation of public-private partnerships, and a combination of tax incentives, regulatory reforms and outreach initiatives. The price tag is estimated to be $10 billion/year. The policy is expected to [decrease / increase] the percentage of Americans living [too far from / close enough to] hospitals from [20% to 10% / 80% to 90%] by 2025. Overall, the design was a 4x2x2 factorial—questions regarding four topic areas, each framed either negatively or positively, and either focused on reactions to problems or reactions to policy solutions. Respondents were randomized into one of the 16 possible conditions. **Outcomes** *Problem Dependent Variables* For each of the dependent variables shown below, the order of the response options was randomized (e.g. Not angry – Extremely angry/Extremely angry – Not angry). The order in which the four questions were asked was also randomized. Anger How angry does this situation make you? Not angry - A little angry - Somewhat angry - Fairly angry - Quite angry - Very angry - Extremely angry Worry How worried are you about this problem? - Not worried - A little worried - Somewhat worried - Fairly worried - Quite worried - Very worried - Extremely worried Serious In your opinion, how serious of a problem is this situation? - Not serious - A little serious - Somewhat serious - Fairly serious - Quite serious - Very serious - Extremely serious Priority How much priority should the U.S. government give to improving this situation? - Lowest priority - Low priority - Moderate priority - Moderately high priority - High priority - Very high priority - Highest priority Plus an 8th option always placed below others: “Government should not be involved in this area.” <br> *Policy Dependent Variables* For each of the dependent variables shown below, the order of the response options was randomized (e.g. Strongly Disapprove to Strongly Approve vs. Strongly Approve to Strongly Disapprove). The order in which the questions were presented was fixed, as given below. Opinion What is your view of this policy proposal? - Strongly Disapprove - Somewhat Disapprove - Slightly Disapprove - Neither Approve nor Disapprove - Slightly Approve - Somewhat Approve - Strongly Approve Petition Would you be willing to support a petition in support of this policy? - Definitely would not - Very probably would not - Probably would not - Might or might not - Probably would - Very probably would - Definitely would Volunteer One element of the outreach is to engage citizen volunteers in efforts to solve the problem. Would you be willing to lend your time and effort to this effort? - Definitely would not - Very probably would not - Probably would not - Might or might not - Probably would - Very probably would - Definitely would <br> *Other Variables* In addition to the core problem or policy DV questions, we also asked a standard set of demographics, plus questions concerning party identification. We also asked questions to measure two potential moderators: Risk Attitude and Need for Cognition. These questions were asked at the start of the survey, prior to the experiment. After the last survey question, we offered an optional but full debriefing. The debriefing page included study information as well as source information and links for each of the problem depictions. **Summary of Results** *Problem Experiments* Results were in the expected direction across all four experiments and all four dependent variables (DVs). Negatively framed problems tended to generate more anger and worry, and to be judged more serious and more worthy of government attention. In 10 of the 16 comparisons (four issues, four DVs each), the treatment effect was substantively and statistically significant (p<.05, two-tailed). If the data for the four experiments were stacked and analyzed with fixed effects for problem area and DV order randomizations, treatment effects were significant for all four dependent variables at p<.001. However, the effect size was significantly larger for Anger (b=.66, with Y scored 0-1, beta=.17) than for Worry (b=.44, beta=.12), Seriousness (b=.44, beta=.13), and Government Priority (b=.26, beta=.08). There was some evidence of anger carry-over effects, in that the effect sizes for Worry, Seriousness, and Government Priority were larger if the respondent was first asked about the extent to which the problem made him or her angry. Effect sizes were not moderated by Need for Cognition or Risk Attitude. They were larger among college-educated respondents (vs. the less educated) and among political Independents (vs. among Democrats or Republicans), but these differences did not reach conventional levels of statistical significance. *Policy Experiments* Results were consistent across all four experiments and all three dependent variables. In no case was there a substantively significant difference in opinions by frame, and 11 of the 12 tests of framing effects (four issues, three DVs each) yielded a p-value greater than .40. Moreover, we found no evidence of significant effects among subgroups defined by potential moderators (Need for Cognition, Risk Atittude, Education, Age, or Partisanship). **References** Pilot Study Results from a survey of MTurk workers were reported in “Public Opinion on Policy Solutions—the Role of Equivalence Frames, Policy Scope, and Party Cues,” a paper presented at the 2018 American Political Science Association Convention. A paper reporting on both the TESS and MTurk studies will be under review soon.