Participants first watched the introductory video, then completed the information sheet.
Participants were first asked:
What is your personal opinion? Do you think that the world’s temperature probably has been going up over the past 100 years, or do you think this probably has not been happening? [unnumbered 1= Probably has been happening/0 = Probably has not been happening]
On a separate page
If nothing is done to reduce global warming in the future, how serious of a problem do you think it will be for the United States? [5-1 unnumbered Likert scale: Extremely serious, very serious, moderately serious, slightly serious, or not serious at all?]
On the next page, participants were randomly assigned to one of two groups (labeled/unlabled).
Below is what a researcher might say after conducting a survey measuring Americans’ opinions about global warming. Please read the paragraph, and then we'll ask you some questions about it.
This is the paragraph:
“One group of people believed that global warming has been happening, that it has been caused by humans, that it is bad, and that it requires government action. The other group of people**[, climate change deniers,/,]** believed that global warming has not been happening, that if it has been happening, it has not been caused by humans and is not a bad thing, and that government should not do anything about it."
On the next page, participants were asked:
From what you read, do you think the researcher thinks that global warming has been happening, that global warming has not been happening, or can you not tell whether the researcher thinks that global warming has or has not been happening? [unnumbered 1 = Researcher thinks global warming has been happening, 0 = Researcher thinks global warming has not been happening, 2 = Cannot tell whether researcher thinks global warming has or has not been happening
On the next page:
From what you read, who do you think the researcher respects more? People who believe global warming has been happening, people who believe global warming has not been happening, or do you think the researcher respects them both equally? [unnumbered 2 = People who believe global warming has been happening, 0 = People who believe global warming has not been happening, 1 = Respects both equally
On the next page:
From what you read, if nothing is done to reduce global warming in the future, how serious of a problem do you think the researcher thinks it will be for the United States? Extremely serious, very serious, moderately serious, slightly serious, or not serious at all? [Unnumbered Likert scale 5 = Extremely serious, 4 = very serious, 3 = moderately serious, 2 = slightly serious, 1 = not serious at all?]
On the next page:
How much would you trust the things this researcher says about the survey he or she conducted? A great deal, a lot, a moderate amount, a little, or not at all? [5-1 Unumbered Likert 5 = A great deal, 4 = a lot, 3 = a moderate amount, 2 = a little, 1 = not at all?]
On the next page:
How much do you trust researchers who conduct surveys measuring Americans’ opinions about global warming? A great deal, a lot, a moderate amount, a little, or not at all? [ 5-1 Unumbered Likert 5 = A great deal, 4 = A lot, 3 = A moderate amount, 2 = A little, 1 = Not at all].
On the next page, participants will be asked:
To help us be sure that your [device/computer] is working properly with ours, please select “Slightly disagree” below. [Unnumbered, 0 = Strongly agree, 0 = Agree, 0 = Slightly agree, 0 = Neither agree nor disagree, 1 = Slightly disagree, 0 = Disagree, 0 = Strongly disagree]
At the end of the survey, we will take the following demographics:
Are you a male or female? [0 = Male, 1 = Female]
[NEW SCREEN]
What is your date of birth?
Enter birth year [TEXTBOX ALLOWING NUMBERS 1900-1999]
[NEW SCREEN]
What is your date of birth?
Enter birth month: [Jan, Feb, Mar, Apr, May, Jun, Jul, Aug, Sep, Oct, Nov, Dec]
[NEW SCREEN] [SHOW IF BIRTH MONTH JAN, MAR, MAY, JUL, AUG, OCT, DEC]
What is your date of birth?
Enter birth day: [DROPDOWN LIST NUMBERS 1-31]
[NEW SCREEN] [SHOW IF BIRTH MONTH APR, JUN, SEP, NOV]
What is your date of birth?
Enter birth day: [DROPDOWN LIST NUMBERS 1-30]
[NEW SCREEN] [SHOW IF BIRTH MONTH FEB]
What is your date of birth?
Enter birth day: [DROPDOWN LIST NUMBERS 1-29]
[NEW SCREEN]
Are you Spanish, Hispanic, or Latino? [unnumbered 1= Yes, 0 = No]
[NEW SCREEN]
Below please choose one or more races that you consider yourself to be: [ALLOWING MULTIPLE RESPONSES, unnumbered, 0 = White, 1 = Black or African American, 2 = American Indian or Alaska Native, 3 = Asian, 4 = Native Hawaiian or Other Pacific Islander, 5 = Some other race]
[NEW SCREEN]
What is the highest level of school you have completed or the highest degree you have received? [Unnumbered, 0 = Less than 1st grade, 1 = 1st, 2nd, 3rd or 4th grade, 2= 5th or 6th grade, 3 = 7th or 8th grade, 4 = 9th grade, 5 = 10th grade, 6 = 11th grade, 7 = 12th grade NO DIPLOMA, 8 = HIGH SCHOOL GRADUATE- high school DIPLOMA or the equivalent (For example: GED), 9 = Some college but no degree, 10 = Associate degree in college - Occupational/vocational program, 11 = Bachelor's degree (For example: BA, AB, BS), 12 = Master's degree (For example: MA, MS, MEng, MEd, MSW, MBA), 13 = Professional School Degree (For example: MD, DDS, DVM, LLB, JD), 14 = Doctorate degree (For example: PhD, EdD)]
[NEW SCREEN]
The next question is about the total income in 2016 for you and all members of your family who lived with you during 2016, before taxes. Please include money you and all members of your family received from jobs, pensions, social security, interest, dividends, capital gains, profits from businesses, unemployment payments, and all other money received. Adding up the income from all these sources and all other sources, was the total income of you and all members of your family who lived with you in 2016, before taxes, less than $50,000 or was it $50,000 or more? [Unnumbered: 0 = Less than $50,000, 1 = $50,000 or more]
[NEW SCREEN] [SHOW IF INCOME LESS THAN $50,000]
And in which of the following groups was the total income of you and all members of your family who lived with you in 2016, before taxes? [Unnumbered, 0 = Less than $10,000, 1 = $10,000 to less than $20,000, 2 = $20,000 to less than $30,000, 3 = $30,000 to less than $40,000, 4 = $40,000 to less than $50,000]
[NEW SCREEN] [SHOW IF INCOME MORE THAN $50,000]
And in which of the following groups was the total income of you and all members of your family who lived with you in 2016, before taxes? [Unnumbered, 5 = $50,000 to less than $75,000, 6 = $75,000 to less than $100,000, 7 = $100,000 to less than $150,000, 8 = $150,000 or more]
[NEW SCREEN]
What is your five digit zip code at your home? [TEXTBOX – ALLOWING ANY VALID ZIP CODE]
[NEW SCREEN]
Did you complete this survey on a computer or on a mobile device? [Unnumbered, 0 = A computer, 1 = A mobile device]
[NEW SCREEN]
Did you receive an email inviting you to answer this survey, or did you not receive such an email? [Unnumbered, 1 = Did receive an email invitation, 0 = Did not receive an email invitation]
[NEW SCREEN] [SHOW IF DID RECEIVE AN EMAIL INVITATION]
What is the name of the company that invited you to answer this questionnaire? [TEXTBOX]
[NEW SCREEN] [SHOW IF DID NOT RECEIVE AN EMAIL INVITATION]
How were you invited to answer this questionnaire? [TEXTBOX]
[NEW SCREEN]
To thank you for completing this questionnaire, will you receive cash, points that you can use to get gifts or prizes or to buy things or to enter lotteries, or will you be compensated in some other way, or will you not be compensated at all? [Unnumbered, 1 = Cash, 2 = Points to use for gifts, prizes, to buy things, or to enter lotteries, 3 = Compensated in some other way, 4 = Not compensated at all]
[NEW SCREEN] [SHOW IF RECEIVED CASH]
How much money will you be paid for completing this survey? [TEXTBOX ALLOWING ANY NUMBER]
[NEW SCREEN] [SHOW IF RECEIVED POINTS TO USE FOR GIFTS, PRIZES, TO BUY THINGS, OR TO ENTER LOTTERIES]
How many points will you be paid for completing this survey? [TEXTBOX ALLOWING ANY NUMBER]
We plan to collect 1500 participants according to the following criteria:
Specifically, the survey firm should be instructed to download random numbers from the generator in batches of 10,000, with each integer having a random value between 1 and 10,000, using 1 column, decimal numeral system, and having “Generate your own personal randomization right now” checked.
Note, that the sequence generator at random.org is not a valid approach to download random numbers, the survey firm must use the integer generator.
Each number drawn should be appended to one respondent in the full sample, until all respondents has been assigned one number each. Respondents who are assigned even random numbers should be treated as belonging to the sample that is first invited to complete the questionnaire and people who are assigned odd random numbers should be treated as belonging to the second sample.
Respondents in the first sample should be sorted in an ascending order according to the random.org number assigned to each person.
Respondents in the second sample should be sorted in an ascending order according to the random.org number assigned to each person.
Beginning with the first person in the sorted list of first sample respondents, enough respondents should be invited so that 750 completed interviews, with respondents passing the attention check(s), is finished collecting within two weeks of the first invitation sent.
Each day of data collection, the host university will update the survey respondent providing firm with two numbers: (1) the number of completed questionnaires that passed the attention check(s); and (2) the number of completed questionnaires that did not pass the attention check(s).
The survey firm will monitor these numbers and send out additional invitations if necessary in order to achieve 750 completed interviews passing the attention check(s) by the end of the 14th day after the data collection begins,
After 750 respondents from the first sample have completed the questionnaire and passed the attention check(s), the second sample will be invited using the same procedure to yield 750 completed interviews passing the attention check(s) by the end of the 14th day after the data collection begins. None of the respondents in the second sample are allowed to be invited before the first sample has finished collecting and been closed for further collection.
After each wave, labs may have their participant company replace participants who failed an attention check.
River sampling and routers may not be used to obtain respondents for these studies, because those methods do not allow selection of the entire potential sample and randomly splitting it in half before data collection begins.
No panel member invited to complete a survey with final N=1,500 may have completed that same survey or a similar one during any prior rounds of testing.