Main content

Date created: | Last Updated:


Creating DOI. Please wait...

Create DOI

Category: Project

Description: An important yet understudied area of researcher discretion is the use of covariates in statistical analysis. Researchers choose which covariates to include and their choices affect the size and statistical significance of their estimates. How often are researchers exploiting this discretion to achieve statistical significance unbeknownst to reviewers and readers? We use newly available replication data to shed light on this question, focusing primarily on observational studies. We find that, when researchers disclose a bivariate estimate of their key effect, adding covariates leaves it unchanged on average. When researchers do not, however, covariates frequently help them achieve statistical significance, and they do so by increasing the magnitude of the effect rather than reducing standard errors. Although such adjustments are potentially justifiable---to address suppressor variables---the articles did not justify them. Our findings do not demonstrate wrongdoing, but suggest that, because of a lack of transparency, the opportunities may be rife.

License: CC-By Attribution 4.0 International


Loading files...


Recent Activity

Loading logs...

OSF does not support the use of Internet Explorer. For optimal performance, please switch to another browser.
This website relies on cookies to help provide a better user experience. By clicking Accept or continuing to use the site, you agree. For more information, see our Privacy Policy and information on cookie use.

Start managing your projects on the OSF today.

Free and easy to use, the Open Science Framework supports the entire research lifecycle: planning, execution, reporting, archiving, and discovery.