# Non-Reactive Indicators for Meaningless Data in Internet Surveys
Data from a study employing an experimental-like design to empirically test the ability of non-reactive indicators to identify records with low data quality. Findings suggest that careless responses are most reliably identified by questionnaire completion time, but the tested indicators do not allow for detecting intended faking.
Leiner, D. J. (2019). Too Fast, too Straight, too Weird: Non-Reactive Indicators for Meaningless Data in Internet Surveys. Survey Research Methods, 13(3), 229-248. [https://doi.org/10.18148/srm/2019.v13i3.7403][1]
## Replication Data Set Description
The paper "Too Fast, too Straight, too Weird" draws data from 4 different surveys, whereas studies 1 and 2 use "high quality" reference data from the same survey, but split into different groups.
Running the file `R-Indicator-Analysis.R` with R will load the data set `R-DQ.RData` and create an output CSV `R-results.csv` with all the results presented in the paper.
## Data
The data set `R-DQ.RData` contains three variables:
* `di`: Quality indicators (per study), computed via `R-Indicators.R`, one data.frame per study
* `dq`: Original data (studies 1/2) or conditions (study 3/4) due to non-author data ownership
* `QInds`: Description for the quality indicator variables (includes some variables that have been tried but not reported)
The variables `di` and `dq` are names lists of four data.frames, each. `di.S1` contains the quality indicators (for their names see `QInds`) from study 1, for example.
The original data from studies 3 and 4 has _not_ been included as the respective rights belong to third parties (see paper). For these studies only the quality indicators calculated from the original data are included in the data set.
Variables in the data.frames are labelled with the R function comment(), i.e., the following line will return the description of variable `MG01_01a` in the data set from study 1.
comment(dq$S1$MG01_01a)
## Quality Indicators
The file `R-Indicators.R` contains all neccessary code to compute quality indicators from any given data set.
The syntax to compute the quality indicators for S1 and S2, for example, is
if (!require("psych")) {
install.packages("psych")
library(psych)
}
source("R-Indicators.R")
vl = list(
# Variables where an answer could not be omitted
compulsory = c(
"MG01_20","MG01_01","MG01_02","MG01_03","MG01_04","MG01_05",
"MG01_06","MG01_07","MG01_08","MG01_09","MG01_30","MG01_10",
"MG01_11","MG01_12","MG01_13","MG01_14","MG01_15","MG01_16",
"MG01_17","MG01_31","MG01_18","MG01_32"
),
# Variables where an answer could be omitted
optional = c(
"ES01_01","ES02_16","ES02_15","ES02_01","ES02_02","ES02_03",
"ES02_04","ES02_05","ES02_06","ES02_07","ES02_08","ES02_09",
"ES02_10","ES02_11","ES02_12","ES02_13","ES02_14","ES17_16",
"ES17_15","ES17_01","ES17_02","ES17_03","ES17_04","ES17_05",
"ES17_06","ES17_07","ES17_08","ES17_09","ES17_10","ES17_11",
"ES17_12","ES17_13","ES17_14",
"ES03_01","ES18_01","ES18_02",
"ES06_01","ES06_02","ES06_03","ES06_04","ES06_05","ES06_06",
"ES06_07","ES06_08","ES06_09","ES06_10","ES06_11","ES06_12",
"ES06_13",
"ES07_01","ES07_03","ES07_05","ES07_02","ES07_04","ES07_06",
"ES08_01","ES19_02","ES10_02","ES10_03","ES12_01",
"ES11_01","ES11_02","ES11_03","ES11_04","ES13_01","ES13_02",
"ES13_03","ES13_04","ES13_05","ES13_06",
"ES14_01","ES14_02","ES14_03","ES14_04","ES14_05","ES14_06",
"ES14_07","ES14_08","ES14_09","ES14_10","ES15_01",
"ES16_06","ES16_05","ES16_01","ES16_02","ES16_03","ES16_04",
"ES20_01","ES20_02","ES21_01",
"KV06_01","KV05","KV08","KV13_01","KV14","KV16","KV16_12",
"KV17","KV18","KV19"
),
# Open-ended responses
open = c(
"ES04x01","ES04x02","ES04x03","ES04x04","ES04x05","ES04x06",
"ES04x07","ES04x08","ES04x09","ES04x10","ES04x11","ES04x12",
"ES04x13","ES04x14","ES04x15",
"ES04x16","ES04x17","ES04x18","ES04x19","ES04x20","ES04x21",
"ES04x22","ES04x23","ES04x24","ES04x25","ES05x01","ES05x02",
"ES05x03","ES05x04",
"ES05x05","ES05x06","ES05x07","ES05x08","ES05x09","ES05x10",
"ES05x11","ES05x12","ES05x13","ES05x14","ES05x15","ES05x16",
"ES05x17","ES05x18","ES05x19","ES05x20",
"ES05x21","ES05x22","ES05x23","ES05x24","ES05x25"
),
# Checkboxes (multiple choice questions allowing for multiple options to be checked)
checkbox = c(
"ES22_01","ES22_02","ES22_03","ES22_04","ES22_05","ES23_01",
"ES23_02","ES23_03","ES23_04","ES23_05","ES24_01","ES24_02",
"ES24_03","ES24_04","ES24_05",
"ES27_01","ES27_02","ES27_03","ES27_04","ES27_05","ES25_01",
"ES25_02","ES25_03","ES25_04","ES25_05","ES26_01","ES26_02",
"ES26_03","ES26_04"
),
# Prefixes of scale question variables
scales = c(
"ES02", "ES06", "ES07", "ES14", "ES16"
),
# Response times per questionnaire page
times = c(
"DUR001", "DUR002", "DUR003", "DUR004", "DUR005", "DUR006",
"DUR007", "DUR008", "DUR009", "DUR010", "DUR011", "DUR012"
)
)
di$S1 = dcIndicators(dq$S1, vl)
di$S2 = dcIndicators(dq$S2, vl)
[1]: https://doi.org/10.18148/srm/2019.v13i3.7403
