Date created: | Last Updated:

Category: Project

Description: We propose to develop Transparency Check, including software that provides an automatic assessment and suggestions for the improvement of the transparency of data and methods in research reports before they are published. Transparency Check serves persons working for universities and research institutes, professional organizations, academic journals, and funding agencies. Through an open call, we organize a series of collaborative deliberations within communities of researchers using seven different sources of data: materials, registers, observations, surveys, news and social media, interviews and focus groups, and participant observation. Each data community aspires to consensus about a set of aspects of research that should be made transparent, and collects exemplary models of suboptimal and commendable documentation of these aspects. We use the community input to create checklists to assess research quality for each source of data, develop the software, and improve it with feedback from data communities. Transparency Check assesses how Findable, Accessible, Interoperable, and Reusable (FAIR) research materials (measurement instruments, data, software) underlying research reports are. The software extracts and retrieves information from the report itself as well as from external sources that is crucial for reviewers, editors, and peers to evaluate and re-use the central claims in an empirical study in the social sciences and humanities. For a research report, the software provides a diagnosis of the presence and completeness of transparency indicators and visualizes it in a user-friendly dashboard. A monitor presents aggregated dashboard data at the level of research groups, departments, or faculties. We produce seven FAIR datasets of research quality indicators, one for each data community, including descriptions of transparency criteria as well as examples of good documentation and poor documentation of these criteria provided by data community members. We describe the datasets in a data paper. In a software paper we describe the software development and its functionalities.