Main content

Home

Menu

Loading wiki pages...

View
Wiki Version:
This preregistration pertains to an experiment conducted during the 2018 U.S. Midterm elections in partnership with Progress Texas, a progressive organization in Texas. The purpose of the study was to determine if longitudinal exposure to issue-oriented Facebook ads could increase voter turnout among so-called "drop-off" voters in the 2018 U.S. Midterm elections. To conduct the study, Progress Texas obtained a list of **871,479 "drop-off" voters** -- essentially, individuals likely to skip the 2018 Midterms without further intervention. These voters were selected based on age and voting history: all had either voted in the 2016 General Election (or registered after the 2016 General Election) but had not voted in any prior election, and were age 40 or under. The study was restricted to select urban / I-35 corridor counties in Texas: Bexar, Collin, Dallas, Denton, Fort Bend, Harris, Hays, Tarrant, Travis, and Williamson. Treatment: A total 40,000 of subjects were randomly assigned to each of 4 treatment groups: Women's Health, Health Care, Immigration, and Gun Control. The remaining 711,479 subjects were placed in the Control group. This sample size was based on an a priori power analysis for pairwise comparisons between each treatment group and the control group able to detect the following increase in turnout based on baseline / control group turnout: If we have 871,480 subjects and we do 40k per treatment group that gives us the ability to detect a... - 0.55% increase in turnout if baseline in control is 30% - 0.52% increase in turnout if baseline in control is 25% - 0.48% increase in turnout if baseline in control is 20% Each group was uploaded separately to Facebook using the Custom Audience feature and matched to the Facebook user graph with name, address, and birth date; Facebook does not make it possible to see the match rate, or who matched. Thus, analysis will be conducted at the level of assignment. However, is possible to see the total number of subjects who were exposed to each ad, which will be reported in the analysis. The total budget for the study was 25,000 USD. Initially, the group planned to spend 400 USD per ad per group, estimating a 10 dollar CPM, which would be sufficient to reach 40,000 people at least once -- 400 @ 10 per 1000 impressions = 40,000 impressions. Ads were set to run for 7 days. New ads were published every 3 to 4 days. After the first week of the study, the ad bid was raised to $500 to increase the number of subjects who were exposed to the ads. Treatment consisted of content created by Progress Texas and included images / memes, as well as links to internal blog posts, videos, and podcasts. The format of treatment was consistent across groups: for instance, all groups would see ads promoting a podcast featuring content to their issue condition. Treatment duration lasted for 7 weeks (September 18 - November 6, 2018). Each individual ad was set to run for 7 days, with subjects were able to see a maximum of 3 impressions from each ad. New ads went up approximately every 3-4 days, thus providing a constant potential stream of content for subjects. Ads were intended to run on Facebook only, but the partner accidentally exposed some Instagram audiences to treatments early in the experiment. This analysis plan is being created and filed before the researcher has access to data on whether or not the subjects voted. **Data Preparation** Subjects who were not registered in Texas as of Election Day (November 6, 2018) or who changed their registration to move between counties within Texas will be removed before analysis. A chi-square test of independence will be performed to assure no association between assignment to treatment and removal from the dataset. **Measurements** - vote: Binary, did subject vote in 2018 midterm election? - treatment: Five-level categorical variable, to which group was the subject assigned - ads: Binary indicator variable, based on whether subject was assigned to any ads (1) or control (0) - age: age in years as of the end of 2018 - sex: male, female, or unknown - county: where the voter was registered - CD: congressional district where the voter was registered - compCD: whether the congressional district was considered competitive or not, based on Cook rankings - lengthreg: the months since the subject most recently registered to vote in Texas - turnoutscore: Catalist modeling score predicting the probability of the subject turning out in the 2018 Midterms **Analysis Plan** Analysis will be conducted in R using binary logistic regression. *Main effects models:* RQ1: Did assignment to a message stream impact turnout? vote ~ treatment vote ~ treatment + age + sex + county + CD + compCD + lengthreg RQ2: Did assignment to ads (any condition) impact turnout? vote ~ ads vote ~ ads + age + sex + county + lengthregistered *Hypothesized interaction effects:* Electoral Salience: The underlying interest in an election moderates treatment (Arceneaux & Nickerson, 2009; Haenschen & Jennings, 2019; Malhotra et al, 2011). In this study, some of the counties saw competitive countywide elections, and some parts of some counties contained competitive Congressional elections. Ads might have been more effective in competitive districts. RQ3: Did Congressional-level electoral salience moderate the effect of treatment? vote ~ ads + compCD + ads:compCD vote ~ ads + compCD + ads:compCD + age + sex + county + CD + lengthreg vote ~ treatment + compCD + treatment:compCD vote ~ treatment + compCD + treatment:compCD + age + sex + county + CD + lengthreg RQ4: Did county-level electoral salience moderate the effect of treatment? vote ~ ads + county + ads:county vote ~ ads + county + ads:county + age + sex + CD + lengthreg vote ~ treatment + county + treatment:county vote ~ treatment + county + treatment:county + age + sex + CD + lengthreg Voter Propensity: A voter's individual likelihood of voting moderates whether they are susceptible to mobilization. Voters with mid-range (usually 30-60%) turnout scores in a given election are usually considered the best targets for mobilization. RQ5: Did voter propensity moderate the effect of treatment? vote ~ ads + turnoutprop + ads:turnoutprop vote ~ ads + turnoutprop + ads:turnoutprop + age + sex + county + CD + lengthreg vote ~ treatment + turnoutprop + treatment:turnoutprop vote ~ treatment + turnoutprop + treatment:turnoutprop + age + sex + county + CD + lengthreg *Exploratory:* Any significant covariates will be tested for interaction terms with treatments, though there are no hypothesized effects.
OSF does not support the use of Internet Explorer. For optimal performance, please switch to another browser.
Accept
This website relies on cookies to help provide a better user experience. By clicking Accept or continuing to use the site, you agree. For more information, see our Privacy Policy and information on cookie use.
Accept
×

Start managing your projects on the OSF today.

Free and easy to use, the Open Science Framework supports the entire research lifecycle: planning, execution, reporting, archiving, and discovery.