## Supplemental Materials Accompanying the Article "Uncovering Null Effects in Null Fields: The Case of Homeopathy" <a href="https://github.com/NilsPetras/metamix"><img src="https://files.osf.io/v1/resources/wuq2h/providers/osfstorage/64c7c9304f0a44148699443b?mode=render"></a> ### Abstract Sigurdson, Sainani, and Ioannidis discussed homeopathy as a prototypical example of a "null field" where true effects are nonexistent and positive effect sizes reflect bias only. Based on a sample of published randomized placebo-controlled trials, they observed a remarkable effect in favor of homeopathy (Hedges' g = 0.36). The authors concluded that this reflects "the average impact of the bias present in the field". We argue that the estimated amount of bias largely depends on the meta-analytic measure used to quantify treatment effects. By applying a bias-corrected measure instead, we show that the maximum-likelihood estimate of the true effect reduces to virtually zero when selective publishing of significant results is appropriately taken into account. We conclude that inclusion of bias-corrected measures should become routine practice in meta-analyses. In line with Sigurdson, Sainani, and Ioannidis' reasoning, we recommend "null fields" as domains to validate such measures. ### Content of this repository #### Analysis code The analysis code behind the supplementary material can be found on [GitHub](https://github.com/einGlasRotwein/erdfelder_null_effects) as well. `R` code for our analyses can be found in the script `erdfelder_supplement.R`. You can find the results of this script in the file `erdfelder_supplement.html` (the [Quarto](https://quarto.org/) document rendering this report can be found on [GitHub](https://github.com/einGlasRotwein/erdfelder_null_effects)). #### Data In the folder `data`, you can find two data files: 1) The full data set accompanying the Sigurdson et al. article, which our analysis is based on. 2) For convenience, we also uploaded a reduced data set that only contains the subset of studies we analyzed. This subset of the data also contains the t-values we calculated from the Hedges' g reported in the Sigurdson data. See the codebook in the `data` folder for further details. ### metamix The analyses rely on the package `metamix` (described in the article). It can be found on [GitHub](https://github.com/NilsPetras/metamix) and can be installed as follows: ```R library("devtools") # if not available: install.packages("devtools") install_github("NilsPetras/metamix") # load the package via library("metamix") ```