The Right Answer to the Wrong Question: Utility of Fixed-margin Sampling in Network Psychometrics

Date created: | Last Updated:

Identifiers: DOI | ARK

Creating DOI and ARK. Please wait...

Create DOI / ARK

Category: Project

Description: Steinley, Hoffman, Brusco and Sher (2017) recently proposed a new method for evaluating the performance of psychological network models: fixed-margin sampling. The authors investigate LASSO regularized Ising models (eLasso) by generating random datasets with the same margins as the original binary dataset, and conclude that many estimated eLasso parameters are not distinguisha-ble from those that would be expected if the data were generated by chance. We argue that fixed-margin sampling cannot be used in this regard, as it generates data under a particular null-hypothesis: a unidimensional factor model with interchangeable indicators (i.e., the Rasch model). We show this by discussing relevant psychometric literature and performing simulation studies. Results indicate that while eLasso correctly estimated network models and estimated almost no edges due to chance, fixed-margin sampling performed poorly in identifying true effects as “interesting.” We conclude that fixed-margin sampling is not fit to assess the performance of estimated Ising models or many other multivariate psychometric models, but do offer great promise as a nonparametric test for assessing if items may be interchangeable indicators of a latent trait (e.g., the DSM disease model), as well as to evaluate results from regular network models such as social networks or railway networks.

License: CC-By Attribution 4.0 International

This project represents an accepted preprint submitted to PsyArXiv . Learn more about how to work with preprint files. View preprint


Loading files...


Recent Activity

Loading logs...


Start managing your projects on the OSF today.

Free and easy to use, the Open Science Framework supports the entire research lifecycle: planning, execution, reporting, archiving, and discovery.

Create an Account Learn More Hide this message