Home

Menu

Loading wiki pages...

View
Wiki Version:
<h1>Non-Reactive Indicators for Meaningless Data in Internet Surveys</h1> <p>Data from a study employing an experimental-like design to empirically test the ability of non-reactive indicators to identify records with low data quality. Findings suggest that careless responses are most reliably identified by questionnaire completion time, but the tested indicators do not allow for detecting intended faking.</p> <p>Leiner, D. J. (2019). Too Fast, too Straight, too Weird: Non-Reactive Indicators for Meaningless Data in Internet Surveys. Survey Research Methods, 13(3), 229-248. <a href="https://doi.org/10.18148/srm/2019.v13i3.7403" rel="nofollow">https://doi.org/10.18148/srm/2019.v13i3.7403</a></p> <h2>Replication Data Set Description</h2> <p>The paper "Too Fast, too Straight, too Weird" draws data from 4 different surveys, whereas studies 1 and 2 use "high quality" reference data from the same survey, but split into different groups.</p> <p>Running the file <code>R-Indicator-Analysis.R</code> with R will load the data set <code>R-DQ.RData</code> and create an output CSV <code>R-results.csv</code> with all the results presented in the paper.</p> <h2>Data</h2> <p>The data set <code>R-DQ.RData</code> contains three variables:</p> <ul> <li><code>di</code>: Quality indicators (per study), computed via <code>R-Indicators.R</code>, one data.frame per study</li> <li><code>dq</code>: Original data (studies 1/2) or conditions (study 3/4) due to non-author data ownership</li> <li><code>QInds</code>: Description for the quality indicator variables (includes some variables that have been tried but not reported)</li> </ul> <p>The variables <code>di</code> and <code>dq</code> are names lists of four data.frames, each. <code>di.S1</code> contains the quality indicators (for their names see <code>QInds</code>) from study 1, for example.</p> <p>The original data from studies 3 and 4 has <em>not</em> been included as the respective rights belong to third parties (see paper). For these studies only the quality indicators calculated from the original data are included in the data set.</p> <p>Variables in the data.frames are labelled with the R function comment(), i.e., the following line will return the description of variable <code>MG01_01a</code> in the data set from study 1.</p> <pre class="highlight"><code>comment(dq$S1$MG01_01a)</code></pre> <h2>Quality Indicators</h2> <p>The file <code>R-Indicators.R</code> contains all neccessary code to compute quality indicators from any given data set.</p> <p>The syntax to compute the quality indicators for S1 and S2, for example, is</p> <pre class="highlight"><code>if (!require(&quot;psych&quot;)) { install.packages(&quot;psych&quot;) library(psych) } source(&quot;R-Indicators.R&quot;) vl = list( # Variables where an answer could not be omitted compulsory = c( &quot;MG01_20&quot;,&quot;MG01_01&quot;,&quot;MG01_02&quot;,&quot;MG01_03&quot;,&quot;MG01_04&quot;,&quot;MG01_05&quot;, &quot;MG01_06&quot;,&quot;MG01_07&quot;,&quot;MG01_08&quot;,&quot;MG01_09&quot;,&quot;MG01_30&quot;,&quot;MG01_10&quot;, &quot;MG01_11&quot;,&quot;MG01_12&quot;,&quot;MG01_13&quot;,&quot;MG01_14&quot;,&quot;MG01_15&quot;,&quot;MG01_16&quot;, &quot;MG01_17&quot;,&quot;MG01_31&quot;,&quot;MG01_18&quot;,&quot;MG01_32&quot; ), # Variables where an answer could be omitted optional = c( &quot;ES01_01&quot;,&quot;ES02_16&quot;,&quot;ES02_15&quot;,&quot;ES02_01&quot;,&quot;ES02_02&quot;,&quot;ES02_03&quot;, &quot;ES02_04&quot;,&quot;ES02_05&quot;,&quot;ES02_06&quot;,&quot;ES02_07&quot;,&quot;ES02_08&quot;,&quot;ES02_09&quot;, &quot;ES02_10&quot;,&quot;ES02_11&quot;,&quot;ES02_12&quot;,&quot;ES02_13&quot;,&quot;ES02_14&quot;,&quot;ES17_16&quot;, &quot;ES17_15&quot;,&quot;ES17_01&quot;,&quot;ES17_02&quot;,&quot;ES17_03&quot;,&quot;ES17_04&quot;,&quot;ES17_05&quot;, &quot;ES17_06&quot;,&quot;ES17_07&quot;,&quot;ES17_08&quot;,&quot;ES17_09&quot;,&quot;ES17_10&quot;,&quot;ES17_11&quot;, &quot;ES17_12&quot;,&quot;ES17_13&quot;,&quot;ES17_14&quot;, &quot;ES03_01&quot;,&quot;ES18_01&quot;,&quot;ES18_02&quot;, &quot;ES06_01&quot;,&quot;ES06_02&quot;,&quot;ES06_03&quot;,&quot;ES06_04&quot;,&quot;ES06_05&quot;,&quot;ES06_06&quot;, &quot;ES06_07&quot;,&quot;ES06_08&quot;,&quot;ES06_09&quot;,&quot;ES06_10&quot;,&quot;ES06_11&quot;,&quot;ES06_12&quot;, &quot;ES06_13&quot;, &quot;ES07_01&quot;,&quot;ES07_03&quot;,&quot;ES07_05&quot;,&quot;ES07_02&quot;,&quot;ES07_04&quot;,&quot;ES07_06&quot;, &quot;ES08_01&quot;,&quot;ES19_02&quot;,&quot;ES10_02&quot;,&quot;ES10_03&quot;,&quot;ES12_01&quot;, &quot;ES11_01&quot;,&quot;ES11_02&quot;,&quot;ES11_03&quot;,&quot;ES11_04&quot;,&quot;ES13_01&quot;,&quot;ES13_02&quot;, &quot;ES13_03&quot;,&quot;ES13_04&quot;,&quot;ES13_05&quot;,&quot;ES13_06&quot;, &quot;ES14_01&quot;,&quot;ES14_02&quot;,&quot;ES14_03&quot;,&quot;ES14_04&quot;,&quot;ES14_05&quot;,&quot;ES14_06&quot;, &quot;ES14_07&quot;,&quot;ES14_08&quot;,&quot;ES14_09&quot;,&quot;ES14_10&quot;,&quot;ES15_01&quot;, &quot;ES16_06&quot;,&quot;ES16_05&quot;,&quot;ES16_01&quot;,&quot;ES16_02&quot;,&quot;ES16_03&quot;,&quot;ES16_04&quot;, &quot;ES20_01&quot;,&quot;ES20_02&quot;,&quot;ES21_01&quot;, &quot;KV06_01&quot;,&quot;KV05&quot;,&quot;KV08&quot;,&quot;KV13_01&quot;,&quot;KV14&quot;,&quot;KV16&quot;,&quot;KV16_12&quot;, &quot;KV17&quot;,&quot;KV18&quot;,&quot;KV19&quot; ), # Open-ended responses open = c( &quot;ES04x01&quot;,&quot;ES04x02&quot;,&quot;ES04x03&quot;,&quot;ES04x04&quot;,&quot;ES04x05&quot;,&quot;ES04x06&quot;, &quot;ES04x07&quot;,&quot;ES04x08&quot;,&quot;ES04x09&quot;,&quot;ES04x10&quot;,&quot;ES04x11&quot;,&quot;ES04x12&quot;, &quot;ES04x13&quot;,&quot;ES04x14&quot;,&quot;ES04x15&quot;, &quot;ES04x16&quot;,&quot;ES04x17&quot;,&quot;ES04x18&quot;,&quot;ES04x19&quot;,&quot;ES04x20&quot;,&quot;ES04x21&quot;, &quot;ES04x22&quot;,&quot;ES04x23&quot;,&quot;ES04x24&quot;,&quot;ES04x25&quot;,&quot;ES05x01&quot;,&quot;ES05x02&quot;, &quot;ES05x03&quot;,&quot;ES05x04&quot;, &quot;ES05x05&quot;,&quot;ES05x06&quot;,&quot;ES05x07&quot;,&quot;ES05x08&quot;,&quot;ES05x09&quot;,&quot;ES05x10&quot;, &quot;ES05x11&quot;,&quot;ES05x12&quot;,&quot;ES05x13&quot;,&quot;ES05x14&quot;,&quot;ES05x15&quot;,&quot;ES05x16&quot;, &quot;ES05x17&quot;,&quot;ES05x18&quot;,&quot;ES05x19&quot;,&quot;ES05x20&quot;, &quot;ES05x21&quot;,&quot;ES05x22&quot;,&quot;ES05x23&quot;,&quot;ES05x24&quot;,&quot;ES05x25&quot; ), # Checkboxes (multiple choice questions allowing for multiple options to be checked) checkbox = c( &quot;ES22_01&quot;,&quot;ES22_02&quot;,&quot;ES22_03&quot;,&quot;ES22_04&quot;,&quot;ES22_05&quot;,&quot;ES23_01&quot;, &quot;ES23_02&quot;,&quot;ES23_03&quot;,&quot;ES23_04&quot;,&quot;ES23_05&quot;,&quot;ES24_01&quot;,&quot;ES24_02&quot;, &quot;ES24_03&quot;,&quot;ES24_04&quot;,&quot;ES24_05&quot;, &quot;ES27_01&quot;,&quot;ES27_02&quot;,&quot;ES27_03&quot;,&quot;ES27_04&quot;,&quot;ES27_05&quot;,&quot;ES25_01&quot;, &quot;ES25_02&quot;,&quot;ES25_03&quot;,&quot;ES25_04&quot;,&quot;ES25_05&quot;,&quot;ES26_01&quot;,&quot;ES26_02&quot;, &quot;ES26_03&quot;,&quot;ES26_04&quot; ), # Prefixes of scale question variables scales = c( &quot;ES02&quot;, &quot;ES06&quot;, &quot;ES07&quot;, &quot;ES14&quot;, &quot;ES16&quot; ), # Response times per questionnaire page times = c( &quot;DUR001&quot;, &quot;DUR002&quot;, &quot;DUR003&quot;, &quot;DUR004&quot;, &quot;DUR005&quot;, &quot;DUR006&quot;, &quot;DUR007&quot;, &quot;DUR008&quot;, &quot;DUR009&quot;, &quot;DUR010&quot;, &quot;DUR011&quot;, &quot;DUR012&quot; ) ) di$S1 = dcIndicators(dq$S1, vl) di$S2 = dcIndicators(dq$S2, vl)</code></pre>
OSF does not support the use of Internet Explorer. For optimal performance, please switch to another browser.
Accept
This website relies on cookies to help provide a better user experience. By clicking Accept or continuing to use the site, you agree. For more information, see our Privacy Policy and information on cookie use.
Accept
×

Start managing your projects on the OSF today.

Free and easy to use, the Open Science Framework supports the entire research lifecycle: planning, execution, reporting, archiving, and discovery.