<p>Previous work has found that people feel significantly more satisfied with explanations of psychological phenomena when those explanations contain neuroscience information — even when this information is entirely irrelevant to the logic of the explanations. This <em>seductive allure effect</em> was first demonstrated by Weisberg, Keil, Goodstein, Rawson, & Gray (2008), and has since been replicated several times in independent labs (e.g., Fernandez-Duque, Evans, Christian, and Hodges, 2014; Rhodes, Rodriguez, and Shah, 2014). </p>
<p>However, thus far the effect has only been observed with irrelevant neuroscience information added to psychological explanations. It is thus not clear whether the seductive allure effect occurs in the same way in other scientific disciplines, or whether it occurs because of features that are unique to psychology or neuroscience.</p>
<p>The current study aims to test whether the seductive allure effect can be seen for other scientific disciplines and thereby to determine why it happens.</p>
<p>A detailed outline is available in the "Study design" document in the Files section.</p>
<p>Participants will rate the quality of 12 explanations for scientific phenomena (2 each from physics, chemistry, biology, neuroscience, psychology, and social science), half of which are good explanations and half which are bad (non-explanatory). Participants will either read explanations that are drawn from the same discipline as the phenomenon or explanations that reference a more fundamental discipline. Two items (one that the participant rated positively and one that the participant rated negatively) will be re-presented and participants will be asked to justify their responses to these items. They will then complete a modified Cognitive Reflection Test, a set of logical syllogisms, the NSF science literacy scale, and an attitudes towards science questionnaire, in random order. Finally, they will provide demographic information about themselves.</p>
<p>Participants will be recruited from three populations: university undergraduates, Mechanical Turk workers, and individuals with graduate level training in one of the six sciences under investigation.</p>
<p>Power analyses based on effect sizes from previous work suggested an N of 96 per population to achieve 80% power. To ensure that we reach this target after excluding any participants who fail our attention check items, we plan to test 150 participants in each population.</p>
1. Quality: In line with previous investigations of the seductive allure effect, which have found that participants can generally distinguish good from bad explanations, we predict a main effect of explanation quality: Participants will rate good explanations more highly than bad explanations.
2. Irrelevant reductive information: We expect to find that irrelevant information from a more fundamental scientific discipline will sway participants to rate reductive explanations more highly, even though this information provides no additional explanatory power. That is, the seductive allure effect should occur across the sciences, not just for psychology.
3. Discipline: The seductive allure effect may manifest itself in different ways across the six sciences under study. One possibility is that it will be equally strong across all six, which would suggest that people have a general preference for reductive explanations. A different possibility is that the effect will particularly pronounced only in psychology (perhaps because people are intuitively dualist, or because they believe that psychology is not scientific enough and should be explained with reference to neuroscience). A final possibility is that the effect will be more pronounced for several disciplines, in which case an analysis of the common features of these disciplines can provide clues as to why the effect happens.
4. Quality x Reduction interaction: Previous work has found that irrelevant neuroscince information has a particularly strong effect on poor-quality psychology explanations; participants judged these explanations as improving more dramatically in quality with the addition of irrelevant neuroscience information than good-quality explanations. We believe that this interaction effect will occur wherever we see the preference for reductive information, since the poor quality explanations leave the most room for the perceived improvement provided by the irrelevant reductive information.
5. Population: We plan to test both naive subjects and people who are experts in the various disciplines. We predict that naive subjects will show a more pronounced seductive allure effect in general than experts. We also predict that experts will be immune to this effect in their domain of expertise, but that this immunity will not necessarily carry over to other scientific disciplines.
6. Possible moderators: We plan to administer a series of auxiliary measures, including measures ofof participants' logical reasoning abilities and scientific background knowledge. These may influence the seductive allure effect where it occurs. For examples, participants who perform better on the logical syllogisms may be less susceptible to the effect.</p>
<p>A detailed outline is available in the "Analysis Plan" document in the Files section.</p>
<p>Mixed-effects regression models will be used for each subject population to examine the effects on participants' ratings of explanation quality, condition, and scientific domain along with all possible interactions, including random effects of subject and item on the intercept.</p>
<p>Follow-up analyses will examine possible moderating effects of demographic variables and performance on ancillary measures.</p>