Main content

Home

Menu

Loading wiki pages...

View
Wiki Version:
This page contains all the collected study materials and instructions from authors that we have compiled. We also include any comments from other contributors or follow up instructions that we have learned since the beginning of the project. Click "read more" below or choose the "Wiki" option above for further information. For technical problems, please contact OSF help desk (support@osf.io) For questions or information about the studies contact either either Jon Grahe (graheje@plu.edu) or Mark Brandt (m.j.brandt@tilburguniversity.edu) so that the materials can be made available on this website. **Abstract** Influential work on human thinking suggests that our judgment is often biased because we minimize cognitive effort and intuitively substitute hard questions by easier ones. A key question is whether or not people realize that they are doing this and notice their mistake. Here, we test this claim with one of the most publicized examples of the substitution bias, the bat-and-ball problem. We designed an isomorphic control version in which reasoners experience no intuitive pull to substitute. Results show that people are less confident in their substituted, erroneous bat-and-ball answer than in their answer on the control version that does not give rise to the substitution. Contrary to popular belief, this basic finding indicates that biased reasoners are not completely oblivious to the substitution and sense that their answer is questionable. This calls into question the characterization of the human reasoner as a happy fool who blindly answers erroneous questions without realizing it. **Introduction** Decision making is an important part of human life because we make decisions every day. Those decisions can include smaller decisions such as what to eat for breakfast to larger financial decisions that can really impact our lives. However, humans can often make errors in their reasoning (De Neys, Rossi, & Houde, 2013). Humans process the world around them and the decisions they make through two forms of cognitive processing. System 1 processing is the reflexive, intuitive processing that uses pre-established stereotypes and ideas to answer questions while System 2 processing is a more careful, draw out reasoning process that analyzes a situation before making a decision (Sloman, 1996). Often, humans first seek to make a decision or answer a question through System 1 processing while System 2 becomes engaged when there is a disfluency or difficulty within a question (Alter, Oppenheimer, & Epley, 2007). Essentially, the more difficult the question or the less the question flows based off the first reading, the more likely System 2 processing will engage. A common question that demonstrates the two systems is as follows: “A bat and ball cost $1.10. The bat cost $1.00 more than the ball. How much does the ball cost? ___ cents.” (Fredrick, 2005). The quick intuitive response is $0.10. This is an example of System 1 engaging with the question. However, upon closer analysis this quick response is incorrect. With the phrase “more than”, the actual cost of the ball is $0.05. The question requires System 2 processing to reach the correct answer. However, due to the fluency of the statement, System 2 is less likely to engage, and System 1 continues to process the question (Alter et al., 2007). Studies such as Charles and Yeung (2019), show that humans will usually make an initial decision first before evaluating the evidence again to see if there is reason to second-guess their choice. If no evidence is given or is not seen due to the fluency of the question, they proceed while still utilizing the decision made through System 1 processing. However, although System 1 might typically be working, that does not mean that participants are not noticing they have made an error. According to Boldt and Yeung (2015), positive error (Pe), an event-related spike in EEG monitoring when an incorrect choice is perceived to have been made, can be seen in participants. Their study, which compared Pe to the participants’ ranked confidence level with their answer establishes correlation between the two variables on a continuum. When the initial decision is made, we can anticipate that participants will rank their confidence with their answers at a lower level if they perceive they may have made an error. A study (De Neys, Rossi, & Houde, 2013) asked participants a standard and control version of two questions that corresponded with the original bats and balls question and interchanged them per participant. The standard question is designed to elicit unconscious substitution. The control question is designed not to illicit substitution. The researchers asked participants to answer these questions and then rate how confident they were with their answer to these questions. The researchers found that most participants answered the standard question incorrectly and reported low confidence in their answer. Most participants answered the control question correctly and reported a significantly higher confidence rating than for standard question. This study shows that subjects were not completely unaware that they made a substitution. This study titled “Bats, balls, and substitution sensitivity: cognitive misers are no happy fools” has not yet been directly replicated. We aim to replicate this study as closely as possible and compare our results to see if the findings are consistent. 110 studies have since cited De Neys et al. (2013). One example includes a study by Thomas, Didierjean, and Kuhn (2018). They applied the two types of cognitive processing to optical illusions. This study showed that participants used System 1 thinking and substituted an unusual action sequence of events for a more familiar one, and this caused participants fall for the illusion. This study also found that participants were not able to detect that they had made an error in this case (Thomas et al., 2018), however, previous studies have shown that people can detect errors in other cases when using System 1 processing (De Neys, Rossi, & Houde, 2013). Another example of a study using De Neys et al. (2013) by Gangemi, Bourgeois-Gironde, and Mancini (2015) tested participants using the bat and ball problem, but the researchers tested the participants’ feelings of error using a questionnaire developed to test for obsessive compulsive disorder. This study did show that people were aware of when they made an error. It built on previous knowledge by showing that when presented with the correct and incorrect answer, participants choose the incorrect answer due to the answer feeling familiar (Gangemi et al., 2015). Further studies that use De Neys et al. (2013) include: A study by Rossi, Cassotti, Agogue, and De Neys (2013) focused on the development of knowledge of false reasoning. They found that when adolescents were given the problem used by De Neys, Rossi, and Houde (2013) the same proportion of participants answered the question incorrectly as in the original study but also found that the participants had rated their confidence significantly higher than the adults in the original study. Another study conducted by Johnson, Tubau, and De Neys (2015) found that biased reasoners gave lower confidence ratings and were slower to respond. The authors clarify that having less confidence in an incorrect answer and knowing that it is wrong is not the same as knowing the correct answer. De Neys et al.’s 2013 study has been used to explain parts of human reasoning. Bago and De Neys (2016) critique of the Default Interventionist Model which is defined in this paper as System 2 correcting the initial heuristic response given by System 1. Using tasks designed to limit System 2 engagement while later allowing participants to submit a final answer without these restraints, they found that a large number of participants gave the correct response and that very few changed their responses. These findings are in direct contrast with the DI model. It is important to remember that decision making and influences over it impact humans’ lives such as in finance, choosing an answer for multiple-choice tests, and general perceptions. Replication is necessary to make sure the results and the findings of the original study are viable and not just a fluke, especially when a study has been cited as frequently as De Neys et al. (2013). We will utilize a within-subjects experimental design in our direct replication of De Neys et al. (2013). Each participant will receive the standard and control version of the questions and order of the questions will be counterbalanced. Confidence ratings will be given immediately after participants respond to the question. We hypothesize that a majority of participants will give the incorrect response on the standard version and that confidence ratings will be lower for that question. **Materials** The original paper is [here][1] (will require a library subscription). Materials are [here][2] [1]: https://link.springer.com/article/10.3758/s13423-013-0384-5 [2]: https://osf.io/9kb9e/
OSF does not support the use of Internet Explorer. For optimal performance, please switch to another browser.
Accept
This website relies on cookies to help provide a better user experience. By clicking Accept or continuing to use the site, you agree. For more information, see our Privacy Policy and information on cookie use.
Accept
×

Start managing your projects on the OSF today.

Free and easy to use, the Open Science Framework supports the entire research lifecycle: planning, execution, reporting, archiving, and discovery.