Main content

Home

Menu

Loading wiki pages...

View
Wiki Version:
Authors: Joanna Emerson, Rachel Bacon, Alma Kent, Peter J Neumann, Joshua T Cohen Background: Computer simulation models now serve as the analytic bedrock for the vast majority of health economic analyses. The prominence of those analyses continues to grow due to increasing concern over health care costs. Cohen et al. [1] and Cohen and Wong[2] have argued that authors should publish the simulation model “source code” – i.e., the model’s human-readable computer instructions. They explained that releasing the source code would boost model credibility and allow other researchers to adapt existing computer code to answer similar questions, thus increasing the efficiency of the field of health economics. Pedulla et al.[3] have argued against the expectation that authors publish their source code, citing intellectual property concerns and the potential for models to be misused to promote misleading claims. Methods: The clearinghouse: The Center for the Evaluation of Value and Risk in Health (CEVR) at Tufts Medical Center (Boston, MA) has developed an online “Open-Source Model Clearinghouse”. The Clearinghouse, funded by the Bill and Melinda Gates Foundation, lets authors publicly post models, and for anyone else to locate those models and download the code. CEVR asks each author to provide summary information about their model, but otherwise imposes no exclusion criteria. Contacted authors: We contacted primary authors of articles published form 2010 to 2017 describing original cost per Disability-Adjusted-Life-Year (DALY) analyses. The Tufts Global Health CEA Registry (www.GHCEARegistry.org) catalogs all of these articles. Survey: We asked authors if they would post their code in the clearinghouse. We sent an initial email and one reminder email between August 8, 2018 and September 6, 2018. We invited authors answering in the affirmative to actually post their models. We asked authors who said they would not post their code about what concerns factored into their decision. Results: We sent emails to 337 authors and received 89 bounce-back emails, making our final sample 248 distinct authors. We received 18 responses (5.3%). Five authors agreed to post their code in the clearinghouse. Of five, four actually submitted models. One ultimately declined to post code, stating that the simulation model “needs to be refined further before publication.” The 13 responding authors who declined to post their code selected the following reasons for their decision (responses not mutually exclusive): intellectual property concerns (3), need to improve code before releasing (3), and need to document code (6). Authors could also provide open responses; those we received expressed concerns regarding intellectual property, the effort needed to document code and provide technical assistance, and the potential for models to be misused. Discussion: Because our response rate was only 5.3%, we cannot use our survey data directly to identify factors discouraging authors from openly publishing their simulation model source code. But we believe the low response rate is itself a finding. It suggests perhaps that many authors do not want to confront the issue of publishing their source code, or at the very least, they do not see source code publication as an important issue. Given that reproducibility of results and open sharing of methods is a basic principle of science, and given our belief that open publication confers numerous other benefits, we believe that if our survey results are indicative of attitudes among health economists, then our field is in need of a substantial cultural change. At stake is the credibility of health economics.
OSF does not support the use of Internet Explorer. For optimal performance, please switch to another browser.
Accept
This website relies on cookies to help provide a better user experience. By clicking Accept or continuing to use the site, you agree. For more information, see our Privacy Policy and information on cookie use.
Accept
×

Start managing your projects on the OSF today.

Free and easy to use, the Open Science Framework supports the entire research lifecycle: planning, execution, reporting, archiving, and discovery.