Main content

Home

Menu

Loading wiki pages...

View
Wiki Version:
Prediction markets have been proposed as useful tools to aggregate information about researchers’ beliefs about scientific results. In this study we will set up prediction markets for a set of hypotheses tested in DARPA’s Next Generation Social Science (NGS2) program. We will invite researchers to bet on whether the hypothesis will be supported or not. Supported here means that the test result will be in the same direction as the hypothesis, with a p-value less than 0.005. Not supported means any other test result. In addition to betting on this binary outcome, we will ask researchers to bet on the standardized effect size (in Cohen’s d) for each hypothesis. We anticipate that we will recruit about 50 participants to these markets. We will also ask participants to fill out an incentivized survey before the prediction markets start. This survey will be on both the binary result whether the hypothesis is supported or not and the effect size. Our goals from this study are to estimate peer beliefs about the outcomes of the tested hypotheses and to test if prediction markets and incentivized surveys can predict the outcomes of the hypotheses being tested and thus be used as reproducibility indicators in a larger sense. This study will increase our knowledge about the predictability of scientific results and the dynamics of hypothesis testing.
OSF does not support the use of Internet Explorer. For optimal performance, please switch to another browser.
Accept
This website relies on cookies to help provide a better user experience. By clicking Accept or continuing to use the site, you agree. For more information, see our Privacy Policy and information on cookie use.
Accept
×

Start managing your projects on the OSF today.

Free and easy to use, the Open Science Framework supports the entire research lifecycle: planning, execution, reporting, archiving, and discovery.