Date created: | Last Updated:

Category: Project

Description: In their recently published paper, Forbes et al. (2019; FWMK) evaluate the replicability of network models in two studies. In both, FWMK identify considerable replicability issues, concluding that “current ‘state-of-the-art’ methods in the psychopathology network literature […] are not well-suited to analyzing the structure of the relationships between individual symptoms”. Such strong claims require strong evidence, which the authors do not provide. To assess replicability, FWMK analyzed point estimates of network parameters, contrast results of low replicability with the results of two statistical tests we developed that indicate higher replicability, and conclude that these tests are problematic. In this rejoinder, we make four points. First, results of statistical tests are superior to the visual comparison of point estimates, because tests take into account sampling variability and were developed so that researchers stop overinterpreting point estimates. Second, FWMK misinterpret the statistical tests in several important ways. Third, the authors did not follow established conventions in their first study, which artificially reduces replicability. Fourth, the authors draw conclusions about methodology, but these do not and cannot follow from investigations of data—they require investigations of methodology, which can be done via simulation studies or mathematical proofs. Overall, we show that the “limited reliability of the detailed characteristics of networks” observed by FWMK occurs in part due to sampling variability, and in part because of their use of polychoric correlations. We conclude by discussing important recent simulation work that guides researchers to use models appropriate for their data, such as nonregularized estimation routines.