Date created: | Last Updated:

Category: Communication

Description: This project deliverable has been submitted to, but not yet reviewed by the Research Executive Agency, and might thus be subject to change. This report presents the main findings from the work undertaken to in TIER2 Task 4.1 “Future studies to identify priorities from the stakeholder community to predict future of reproducibility and identify actionable steps”. The study aims to determine the future of reproducibility. We do this by exploring potential scenarios of the future, examining what steps are needed to make these scenarios workable in the future and detect barriers and facilitators for reproducibility in the current research system and the future one. In addition, we also aim to identify the key factors and stakeholders that may have the ‘power’ to influence these scenarios. To address these aims, we formulated four research questions: 1. What are the preferred futures of reproducibility? 2. What are the alternative futures: dystopian, realistic, outlier? 3. What are the enablers and barriers on the way to the preferred future or reproducibility more generally? 4. Which stakeholder groups bear responsibility for addressing concerns related to reproducibility and have the ability to trigger change for reproducibility in the future? To answer the questions above, and as presented in the Methodology section of this document, we have conducted four workshops with different stakeholders (publishers, funders, and researchers) from different domains (machine learning and qualitative research). The study employed the method of futures studies, which aims to explore and predict the future by envisioning alternative future scenarios, to inform strategies for action to eventually shape the desired future at different levels. The different components of the workshops drew light on the most important stakeholders who have the greatest influence in fostering change for reproducibility. The themes that emerged from exploring the preferred futures for reproducibility in different stakeholder categories are insightful. Furthermore, we identified enablers that may lead to these futures and the barriers that need to be overcome to achieve these futures. Furthermore, we describe the alternative futures: a dystopian, a realistic and some outlier-scenarios. Participants emphasized the need for a cultural shift towards quality over quantity, with reproducibility becoming the norm. This involves developing common understandings of definitions across disciplines, implementing incentives for reproducible practices, and developing clear guidelines and infrastructures to foster reproducibility. Lack of consensus on terminology, poor levels of awareness, dysfunctional incentives, challenges of epistemic diversity, and the risk of marginalization were identified as the main barriers for cultural change. The outcome of our work is aimed at furthering the discussion on reproducibility. Specifically, the next steps are to use the results in the next stages of the TIER2 project to help develop tools that can enable and facilitate the preferred future. Furthermore, we can examine whether the tools are effective for reaching the preferred futures for the stakeholders. The enablers and barriers identified during the workshops can have a significant influence on the implementation of the pilot tests in the next phase of the project, particularly for helping to successfully implement the tools for each stakeholder group. This can help in steering towards the right direction and being prepared for certain barriers and to help use certain enablers.