Good tests replicate


Date created: | Last Updated:


Creating DOI. Please wait...

Create DOI

Category: Project

Description: What does using a good test mean, and why should psychologists care? From a purely statistical perspective, good tests make practical assumptions about the kinds of data they are applied to, and they perform well when those assumptions are met (e.g., they maintain a 5% false positive error rate). Replication depends in part on whether psychologists are mindful about these assumptions—and the consequences of violating them—when conducting and interpreting test results. Often times, these assumptions are carelessly or unwittingly violated. In this presentation, I will demonstrate how “significant” findings can dramatically mislead and thus also fail to replicate when assumptions are not met. From the perspective of psychological theory, good tests stem from good study design, and their results clearly (dis)confirm specific predictions. Here, replication depends on unambiguous research questions and specific predictions. However, many times tests are used that do not answer the question at hand; tests can be better tailored to specific questions, and tests that better suit research questions are easier to interpret in both original and replication studies. In this presentation, I will give specific examples of how psychologists can increase replication not only by ensuring that their tests meet statistical assumptions but also by ensuring that their research questions easily lend themselves to specific tests.

License: CC-By Attribution 4.0 International


Loading files...



  • slides

    slides from the presentation

    Recent Activity

    Loading logs...

  • R code

    code used to generate plots and data in presentation examples

    Recent Activity

    Loading logs...


Recent Activity

Loading logs...

OSF does not support the use of Internet Explorer. For optimal performance, please switch to another browser.
This website relies on cookies to help provide a better user experience. By clicking Accept or continuing to use the site, you agree. For more information, see our Privacy Policy and information on cookie use.

Start managing your projects on the OSF today.

Free and easy to use, the Open Science Framework supports the entire research lifecycle: planning, execution, reporting, archiving, and discovery.