We will have participants complete a the verbal reflection test (Sirota, Kostovičová, Juanchich, Dewberry, & Marshall, 2018) and test the effect of thinking aloud (Ericsson, 2003) on reflection test performance. Participants will be randomly assigned to either a think aloud condition or a control condition. We hypothesize that think aloud verbal reports of verbal reflection test performance will reveal false positives (e.g., immediately correct responses) and false negatives (i.e., thoughtful incorrect responses). Second, we hypothesize that the thinking aloud condition will not hinder reflection test performance—i.e., it will either not impact performance or it will result in more reflective performance. <h1>Method</h1> **Participants**. People will be recruited from public spaces on a university campus in the Southeastern United States. Participants who completed the experiment will be given the option to enter a sweepstakes to win prizes like a smart speaker, waterbottle, or book. To satisfy rules of thumb about minimum statistical power while also being realistic about our ability to record inviduals thinking aloud at scale, we hope to recruit at least 100 participants, or 50 per condition (Simmons, Nelson, & Simonsohn, 2013). **Follow-up and replication**. If possible, an online verbal report collection protocol will be developed to test whether the findings of Experiment 1 will replicate uding an entirely online protocol. <h2>Procedure</h2> **Manipulation**. Participants will be randomly assigned to either think aloud while completing a reflection test or complete the reflection test on their own. Participants assigned to the think aloud condition will be given instructions about thinking aloud from a researcher. After participants have had a chance to ask for clarification, they will click a button labeled “I received and understand the instructions from the researcher.” Participants will then practice thinking aloud during a single survey item: “To practice thinking aloud, please say this sentence aloud, followed by the following number ….”. The number each participant reads aloud will be generated randomly and used to anonymously pair each think aloud recording with its corresponding survey responses. **Recording think aloud verbalizations.** In the first experiment, recordings will be made using researchers' smartphones. The hope is to create an online verbal report recording protocol for follow-up experiments and/or replications. **Open science goals**. Data and analysis scripts will be shared with reviewers upon submission to journals and with the public upon notification of acceptance for publication. <h2>Materials</h2> **Verbal reflection test**. Participants completed the 10-item verbal reflection test (Sirota, Kostovičová, Juanchich, Dewberry, & Marshall, 2018). One example of these items is as follows, “Mary’s father has a total of five daughters: Nana, Nene, Nini, Nono, and ____. What is the name of the fifth daughter, probably?” Reflective scores will be computed by summing correct responses (e.g., Mary) and unreflective scores will be computed by summing lured responses (e.g., Nunu) on these verbal reasoning items will be summed. **Questions about lures**. To test the default-interventionist account of reflection testing, participants will be asked whether the lured response occurred to them after they submitted their answer to each reflection test question. For instance, after answering the question about Mary’s father above, participants will be asked, “Have you thought at any point that 'Nunu' could be the answer?” <h1>Analysis</h1> **Rating think aloud recordings**. Research assistants will then complete worksheets with questions about whether each response involved verbalizing of (a) a re-consideration of an initial response and (b) reasons for or against any response. **Determining standard coding scheme's false positive and false negative rates**. Ratings of think aloud recordings (and correlations between raters answers) will be used to determine the rate of false positive and false negative reflection test codings of the conventional reflection test coding scheme (i.e., correct as "reflective" and lured as "incorrect). **Determining the impact of thinking aloud**. We will compare verbal reflection test performance between the think aloud and control groups to see if thinking aloud had an impact on verbal reflection test performance. Other questions about the think aloud verbalizations may be adapted from Szaszi et al. (2017) as needed. <h1>References</h1> Ericsson, A. (2003). Valid and Non-Reactive Verbalization of Thoughts During Performance of Tasks Towards a Solution to the Central Problems of Introspection as a Source of Scientific Data. Journal of Consciousness Studies, 10(9–10), 1–18. Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2013). Life after P-Hacking. Meeting of the Society for Personality and Social Psychology, 38. papers.ssrn.com/abstract=2205186 Sirota, M., Kostovičová, L., Juanchich, M., Dewberry, C., & Marshall, A. C. (2018). Measuring Cognitive Reflection without Maths: Developing and Validating the Verbal Cognitive Reflection Test. DOI: 10.31234/osf.io/pfe79 Szaszi, B., Szollosi, A., Palfi, B., & Aczel, B. (2017). The cognitive reflection test revisited: Exploring the ways individuals solve the test. Thinking & Reasoning, 23(3), 207–234. DOI: 10.1080/13546783.2017.1292954