**Module Description**
One common practice in psychological science is "optional stopping" or analyzing data as it comes in and stopping data collection if criteria are met. We discuss problems with this practice and present methods which can alleviate these issues.
**Learning Objectives**
- What is optional stopping?
- What are the problems associated with optional stopping?
- Define sequential analysis.
- Discuss how optional stopping affects results differently when used in frequentist vs. Bayesian statistical analyses.
**Readings**
Sagarin, Ambler, & Lee (2014). [An Ethical Approach to Peeking at data.][1] *Perspectives on Psychological Science*, 9(3), 293-304.
Schonbrodt, F. D., Wagenmakers, E., Zehetleitner, Z., & Perugini, M. (2017). [Sequential Hypothesis testing with Bayes factors: Efficiently testing mean differences.][2] *Psychological Methods*, 22(2). 322-339.
Lakens, D. (2014). [Performing high-powered studies efficiently with sequential analyses: Sequential
analyses][3]. *European Journal of Social Psychology*, 44(7), 701–710.
Rouder, J. N. (2014). [Optional stopping: No problem for Bayesians][4]. *Psychonomic Bulletin & Review*, 21(2), 301-308.
**Blogs**
Lakens, D. (2014). [Data peeking without p-hacking.][5] *The 20% Statistician.*
Yarkoni, T. (2010). [The capricious nature of p < .05, or why data peeking is evil.][6] *[citation needed]*
Srivastava, S. (2012). [Data peeking is always wrong, except when you do it right.][7] *The Hardest Science*
Vazire, S. (2015). [Guest post: Excuses for data peeking.][8] *sometimes i'm wrong*
**Demonstrations**
Strube, M. J. (2006). [SNOOP: A program for demonstrating the consequences of premature and repeated null hypothesis testing.][9] *Behavior Research Methods*, *38*(1), 24-27. (Currently not available online. Must request from author.)
Data peeking slides available in Demonstration component.
[1]: https://www.researchgate.net/profile/Brad_Sagarin/publication/275477618_An_Ethical_Approach_to_Peeking_at_Data/links/565b3cdb08ae1ef92980e9d7.pdf
[2]: https://osf.io/w3s3s/download?format=pdf
[3]: https://osf.io/preprints/psyarxiv/9yegd/download?format=pdf
[4]: https://pdfs.semanticscholar.org/0f7b/f686ed6a052fca9465698acec9e26d7be7e2.pdf
[5]: http://daniellakens.blogspot.com/2014/06/data-peeking-without-p-hacking.html
[6]: https://www.talyarkoni.org/blog/2010/05/06/the-capricious-nature-of-p-05-or-why-data-peeking-is-evil/
[7]: https://hardsci.wordpress.com/2012/11/08/data-peeking-is-always-wrong-except-when-you-do-it-right/
[8]: http://sometimesimwrong.typepad.com/wrong/2015/06/guest-post-excuses-for-data-peeking.html
[9]: https://www.researchgate.net/profile/Michael_Strube/publication/6966456_SNOOP_A_program_for_demonstrating_the_consequences_of_premature_and_repeated_null_hypothesis_testing/links/5513615e0cf2eda0df2f62ec/SNOOP-A-program-for-demonstrating-the-consequences-of-premature-and-repeated-null-hypothesis-testing.pdf
