**Guidelines for Transparency and Openness Promotion (TOP) in Journal Policies and Practices “The TOP Guidelines”** *Version 1.0.1* There are eight transparency standards covered by these guidelines. The guidelines are modular so they can be adopted singly or collectively: @[toc] Each category presents template text for three levels of transparency: Level 1, Level 2, and Level 3. Adopting journals select among the levels based on readiness to adopt milder to stronger transparency standards for authors and researchers. There are many factors that will influence level selection including considerations for implementation, and concordance with disciplinary norms and expectations. Defining distinct standards and multiple levels facilitates journals’ making informed decisions among the possibilities. Also, it acknowledges and embraces the variation in evolving norms about research transparency in the present science ecosystem. The TOP Guidelines Committee, sponsored by the [Center for Open Science][1], maintains an [information commons][2] for transparency standards, serves as an advisory group for journals and funders, evaluate guidelines’ effectiveness, and manages guideline updating to maximize quality and interdisciplinary applicability. Updates to standards are recorded with version number and date. Adopting journals and funders can denote the version number that they adopt to facilitate tracking and updating of standards over time. The remainder of this document provides template guideline text for Level 1, 2, and 3 transparency standards for each category. The Appendix provides the full-text template for each level, and the [TOP Guidelines Summary table][3] provides an accessible overview of the Levels as well as a comparison of common journal policies that do not meet the transparency standards. For information on background and rationale, see the forthcoming report by the contributors to the first [TOP Guidelines meeting on November 3-4, 2014][4]. ---------- 1 Citation Standards --------------------- *Summary:* Citation of articles is routine and well-formulated. Similar standards can be applied to citation of data, code, and materials to recognize and credit these as original intellectual contributions. Level 1 recommends citation standards, Level 2 requires adherence to citation standards, and Level 3 requires and enforces adherence to citation standards. *Level 1* All data, program code and other methods should be appropriately cited. Such materials should be recognized as original intellectual contributions and afforded recognition through citation. - All data sets and program code used in a publication should be cited in the text and listed in the reference section. - References for data sets and program code should include a persistent identifier, such as a Digital Object Identifier (DOI). Persistent identifiers ensure future access to unique published digital objects, such as a text or data set. Persistent identifiers are assigned to data sets by digital archives, such as institutional repositories and partners in the Data Preservation Alliance for the Social Sciences (Data-PASS). - *Data set citation examples:* - Campbell, Angus, and Kahn, Robert L. ANES 1948 Time Series Study. Ann Arbor, MI: Inter-university Consortium for Political and Social Research [distributor], 2015-11-10. https://doi.org/10.3886/ICPSR07218.v4 - Kidwell, M., Lazarevic, L. B., Baranski, E., Hardwicke, T. E., Piechowski, S., Falkenberg, L.-S., … Nosek, B. A. (2016, August 18). Badges to Acknowledge Open Practices: A Simple, Low Cost, Effective Method for Increasing Transparency. Retrieved from osf.io/rfgdw *Level 2* Same text as Level 1 with additions in **bold** (full text in appendix): All data, program code and other methods **must** be appropriately cited. Such materials **are** recognized .... - All data sets and program code used in a publication **must be** cited... - References for data sets and program code must ... *Level 3* Same text as Level 2 with additions in bold (full text in appendix): ... materials are recognized as original intellectual contributions and afforded recognition through citation. **Articles will not be published until the citations conform to these standards.** 2, 3, 4 Data, Analytic Methods (Code), and Research Materials Transparency ------------------------------------------------------------------------ Summary: Transparency guidelines for data, analytic methods, and research materials are conceptually distinct. They are presented together as the process principles are similar for each. However, a journal could adopt different levels for each with minor modifications of the templates. For Level 1, the published article states whether or not data, code, materials are available and, if available, how to access. For Level 2, data, code, materials must be posted to a trusted repository and exceptions to sharing identified at article submission. Level 3 adds independent verification of reported analyses to the Level 2 standard. *Level 1* The policy of the ___________ is to publish papers where authors indicate whether the data, methods used in the analysis, and materials used to conduct the research will be made available to any researcher for purposes of reproducing the results or replicating the procedure. Authors must, in acknowledgments or the first footnote, indicate if they will or will not make their data, analytic methods, and study materials available to other researchers. If an author agrees to make materials available, the author must specify where that material will be available. *Level 2* The policy of the ___________ is to publish papers only if the data, methods used in the analysis, and materials used to conduct the research are clearly and precisely documented and are maximally available to any researcher for purposes of reproducing the results or replicating the procedure. 1. Authors reusing data available from public repositories must provide program code, scripts for statistical packages, and other documentation sufficient to allow an informed researcher to precisely reproduce all published results. 2. Authors using original data must: a. make the data available at a trusted digital repository (Note: If all data required to reproduce the reported analyses appears in the article text, tables, and figures then it does not also need to be posted to a repository.) b. include all variables, treatment conditions, and observations described in the manuscript c. provide a full account of the procedures used to collect, preprocess, clean, or generate the data d. provide program code, scripts, codebooks, and other documentation sufficient to precisely reproduce all published results e. provide research materials and description of procedures necessary to conduct an independent replication of the research 3. In rare cases, despite authors’ best efforts, some or all data or materials cannot be shared for legal or ethical reasons. In such cases, authors must inform the editors at the time of submission. This will be taken into account during the review process. Authors are encouraged to anticipate data and material sharing at the beginning of their projects to provide for these circumstances. It is understood that in some cases access will be provided under restrictions to protect confidential or proprietary information. Editors may grant exceptions to data and material access requirements provided authors: a. explain the restrictions on the dataset or materials and how they preclude public access. b. provide a public description of the steps others should follow to request access to the data or materials. c. provide software and other documentation that will precisely reproduce all published results. d. provide access to all data and materials for which the constraints do not apply. 4. Data, program code, research materials, and other documentation of the research process should be made available through a trusted digital repository. Trusted repositories adhere to policies that make data discoverable, accessible, usable, and preserved for the long term. Trusted repositories also assign unique and persistent identifiers. For example these services are offered by partners in the Data Preservation Alliance for the Social Sciences (Data-PASS) and most institutional repositories. Author maintained websites are not compliant with this requirement. a. Dissemination of these materials may be delayed until publication. Under exceptional circumstances, editors may grant an embargo of the public release of data for at most one year after publication. b. Articles accepted for publication will not be assigned a publication date until the above conditions have been met. Authors are responsible for ensuring that their articles continue to meet these conditions. Failure to do so may lead to an editorial expression of concern or retraction of the article. *Level 3* Same text as Level 2 with additions in bold (full text here): [*opening*] … are maximally available to any researcher for purposes of reproducing the results or replicating the procedure. **All materials supporting the claims made by the author must be made available to the journal prior to publication. The journal, or an entity acting on behalf of the journal, will verify that the findings are replicable using the author’s data and methods of analysis. Failure to replicate at this stage may result in the paper not being published.** 5 Design and Analysis Transparency ----------------------------------- Standards for reporting research design and analysis should maximize transparency about the research process and minimize potential for vague or incomplete reporting of the methodology. The standards for data, analytic methods, and research materials above provide general guidelines for making such material available. This section addresses what should be contained in such disclosures. Standards for the reporting of research design are highly discipline-specific. Thus, we encourage journals to incorporate existing standards that apply to specific domains. To aid implementation, we provide the following outline rather than discipline-specific disclosures: **Principles of disclosure:** 1. Readers can easily identify the reporting standards a paper was subjected to (e.g., someone in 2019 reading a 2015 paper should know what standards were applied to that paper). 2. Readers can easily identify what an individual paper is disclosing (namely: what was disclosed is visible to readers, not just reviewers, in an easy to find format) 3. During the review process, compliance is easy to monitor and enforce with initial submission and further revisions, including between final acceptance and published version. 4. It is difficult for an author to unintentionally fail to disclose (readers should reasonably conclude a failure to disclose was intentional or an act of grave sloppiness). **Defining what to ask for:** 1. Identify the types of research design and analytic methods published in the journal (e.g., between subject lab-experiments, observational studies, network analysis, fMRI studies) 2. Identify if existing reporting guidelines apply to the report (i.e., by browsing standards available for many research applications from http://www.equator-network.org/). 3. Select the guideline(s) that is most relevant to the report 4. If relevant standards do not exist, then a simple default relevant for many fields may be: "All excluded observations, independent variables/manipulations, and dependent variables/measures must be reported, and authors should be sure to explain how the sample size was determined" as used by Psychological Science (see guidelines under "Method" [here][5]). 5. Alternatively, establish or request help to establish standards for research applications not yet represented at EQUATOR **Guidance on how to ask for it** We recommend implementing disclosure standards by having authors submit a form alongside the paper where they answer questions regarding research design. The answers to these questions are: (a) sent to reviewers (b) revised alongside the paper in every round, (c) published alongside the paper (as appendix or online supplement). For concrete implementation of this form: 1. Journals have different disclosure forms for different types of research (e.g., fMRI vs observational study). 2. Authors begin by identifying the type of design of their study and hence the appropriate form to submit 3. Authors document compliance by copy-pasting the relevant passages in the paper that address the question into the form. For example, when indicating how sample size was determined, authors copy paste into the form the text in the paper that describes how sample size was determined. *Note: The template guidelines do not reflect idiosyncratic standards that may be adopted by journals following the guidelines above. Also, if a subset of standards are highly relevant for the types of research published in the journal, then those standards should be part of the submission process following the concrete implementation suggestions above.* *Level 1* The policy of the ___________ is to publish papers where authors follow standards for disclosing key aspects of the research design and data analysis. Authors are encouraged to review the standards available for many research applications from http://www.equator-network.org/ and use those that are relevant for the reported research applications. *Level 2* Same text as Level 1 with additions in **bold** (full text in appendix): … aspects of the research design and data analysis. Authors are **required** to review the standards available for many research applications from http://www.equator-network.org/ and use those that are relevant for the reported research applications. **At manuscript submission, authors must confirm that they reviewed the standards, report whether any standards were relevant for the research application, and confirm that they followed those standards in the manuscript.** *Level 3* Same text as Level 2 with additions in **bold** (full text in appendix): … confirm that they followed those standards in the manuscript. **The journal, or an entity acting on behalf of the journal, will verify that the appropriate standards were adopted and followed. Failure to follow the relevant standards may result in the paper not being published.** 6 Preregistration of Studies ----------------------------- *Summary: Preregistration of studies is a means of making research more discoverable even if it does not get published. By encouraging or requiring preregistration, journals increase the likelihood of discoverability of research that is not ultimately published. For Level 1, journals encourage preregistration, and require links in text to preregistrations of studies if they exist. For Level 2, journals verify that the preregistration follows standards and indicates certification of meeting those standards. In addition to Level 2 guidelines, Level 3 requires that all reported studies were preregistered.* *Level 1* The policy of the ___________ is to publish papers where authors indicate whether the conducted research was preregistered in an independent, institutional registry (e.g., http://clinicaltrials.gov/, http://socialscienceregistry.org/, http://osf.io/, http://egap.org/design-registration/, http://ridie.3ieimpact.org/). Preregistration of studies involves registering the study design, variables, and treatment conditions prior to conducting the research. 1. Authors must, in acknowledgments or the first footnote, indicate if they did or did not preregister the research in an independent, institutional registry. 2. If an author did preregister the research, the author must confirm that the study was registered prior to conducting the research with links to the time-stamped preregistrations at the institutional registry, and that the preregistration adheres to the disclosure requirements of the institutional registry or those required for the [preregistered badge maintained by the Center for Open Science.][6] *Level 2* Same text as Level 1 with additions in **bold** (full text in appendix): [*opening*] … prior to conducting the research. **A link to the preregistration in an institutional registry must be made available to the journal prior to publication. The journal, or an entity acting on behalf of the journal, will verify that preregistration adheres to the specifications for preregistration and then provide [certification of the preregistration][7] in the article.** *Level 3* Same text as Level 2 with additions in **bold** (full text in appendix): The policy of the ___________ is to publish papers **only if** the conducted research was preregistered in an independent, institutional registry … 7 Preregistration of Analysis Plans ------------------------------------ *Summary: Preregistration of studies is a means of making research more discoverable even if it does not get published. Preregistration of Analysis Plans certifies the distinction between confirmatory and exploratory research. Preregistration of Analysis Plans supercedes Preregistration of Studies above. If a transparency standard for analysis plans is adopted, then the text below is adopted instead of text in Preregistration of Studies. An exception to this rule is if a stronger transparency standard is adopted for studies than for analysis plans. In that case, minor edits of the text below may be needed to avoid competing language with above. For Level 1, journals encourage preregistration with analysis plans, and require links in text to preregistrations if they exist. For Level 2, journals verify that the preregistration follows standards and indicates certification of meeting those standards. In addition to Level 2 guidelines, Level 3 requires that all reported studies were preregistered. This may include [Registered Reports][8] as a submission option for preregistered peer review.* *Level 1* The policy of the ___________ is to publish papers where authors indicate whether or not the conducted research was preregistered with an analysis plan in an independent, institutional registry (e.g., http://clinicaltrials.gov/, http://socialscienceregistry.org/, http://osf.io/, http://egap.org/design-registration/, http://ridie.3ieimpact.org/). Preregistration of studies involves registering the study design, variables, and treatment conditions. Including an analysis plan involves specification of sequence of analyses or the statistical model that will be reported. 1. Authors must, in acknowledgments or the first footnote, indicate if they did or did not preregister the research with or without an analysis plan in an independent, institutional registry. 2. If an author did preregister the research with an analysis plan, the author must: a. confirm in the text that the study was registered prior to conducting the research with links to the time-stamped preregistrations at the institutional registry, and that the preregistration adheres to the disclosure requirements of the institutional registry or those required for the [preregistered badge with analysis plans maintained by the Center for Open Science][9]. b. report all pre-registered analyses in the text, or, if there were changes in the analysis plan following preregistration, those changes must be disclosed with explanation for the changes. c. Clearly distinguish in text analyses that were preregistered from those that were not, such as having separate sections in the results for confirmatory and exploratory analyses. *Level 2* Same text as Level 1 with additions in **bold** (full text in appendix): [*opening*] … or the statistical model that will be reported. **A link to the preregistration in an institutional registry must be made available to the journal prior to publication. The journal, or an entity acting on behalf of the journal, will verify that preregistration adheres to the specifications for preregistration and then [provide certification of the preregistration][10] in the article.** *Level 3* Same text as Level 2 with additions in **bold** (full text in appendix): The policy of the ___________ is to publish papers **only if** the conducted research was preregistered with an analysis plan … *Note: Level 3 may also include using [Registered Reports][11] as a submission option or requirement for replications to obtain peer review prior to observing the study outcomes. Model text for those guidelines appears in Level 3 of replications below and is applicable more broadly by removing reference to replication. Also, this guideline does not restrict authors to only planning or conducting confirmatory analyses (i.e., specification of the model in advance and then only reporting the outcomes of that model). A preregistered analysis plan may explicitly state that the analysis strategy is exploratory, and this would likewise be made clear in the published article.* 8 Replication -------------- *Summary: The transparency standards above account for reproducibility of the reported results based on the originating data, and for sharing sufficient information to conduct an independent replication. While not formally a transparency standard for authors, this section addresses journal guidelines for consideration of independent replications for publication. Level 1 encourages submission of replication studies. Level 2 conducts results blind review of replication studies. Level 3 uses [Registered Reports][12] as a submission option for replications to obtain peer review prior to observing the study outcomes.* *Level 0* The journal explicitly discourages submission of replication studies, or says nothing about replications in its guidelines. *Level 1* The policy of the ___________ is to encourage submission of replication studies, particularly of research published in this journal. Note: If the journal will publish replication studies in a separate section, or as online-only in case of print journals, that should be made explicit. *Level 2* The policy of the ___________ is to encourage submission of replication studies, particularly of research published in this journal. Replication studies are normally reviewed in two stages, the first being results-blind. 1. On initial submission, authors should: a. Denote in the cover letter that the manuscript is a replication study submission b. Include a full manuscript for the abstract, introduction, and methods without the results and discussion sections. The submitted manuscript must not indicate information about the outcome-relevant results. c. The methods must contain a complete analysis plan of what is to be included in the full article. d. If relevant, outcome-irrelevant results can be reported to demonstrate, for example, that that experimental manipulations were effective, or outcome variables were measured reliably and conformed to distributional assumptions. 2. If the submission passes initial review, then the authors will submit a complete manuscript for second stage review to confirm that the final report adequately addresses reviewer concerns from the initial submission. *Level 3* The policy of the ___________ is to encourage submission of replication studies, particularly of research published in this journal. When possible, replication studies are reviewed in two stages following the [Registered Reports][13] format. In particular, the first stage of review is conducted prior to the data being collected or, for existing datasets, before the outcomes are observed: 1. On initial submission, authors should: a. Denote in the cover letter that the manuscript is a Registered Report submission and confirm that the data do not exist, or that the outcomes have not been observed b. Include a full manuscript for the abstract, introduction, and methods without the results and discussion sections. c. The methods must contain a complete analysis plan of what is to be included in the full article. 2. If the submission passes initial review, then the authors will receive an in-principle acceptance prior to data collection or analysis of the outcomes. 3. For the second stage of review, authors submit a complete manuscript. Reviewers assess the extent to which the authors followed the preregistered design and/or analysis plan, and evaluate non-outcome relevant criteria (e.g., manipulation checks) to confirm whether the research was an effective test of the research question. ---------- Appendix: Full-text of each level of each transparency standard --------- **1. Citation Standards** *Level 1* All data, program code and other methods should be appropriately cited. Such materials should be recognized as original intellectual contributions and afforded recognition through citation. a. All data sets and program code used in a publication should be cited in the text and listed in the reference section b. References for data sets and program code should include a persistent identifier, such as a Digital Object Identifier (DOI). Persistent identifiers ensure future access to unique published digital objects, such as a text or data set. Persistent identifiers are assigned to data sets by digital archives, such as institutional repositories and partners in the Data Preservation Alliance for the Social Sciences (Data-PASS). c. Data set citation example: Campbell, Angus, and Robert L. Kahn. American National Election Study, 1948. ICPSR07218-v3. Ann Arbor, MI: Inter-university Consortium for Political and Social Research [distributor], 1999. http://doi.org/10.3886/ICPSR07218.v3 *Level 2* All data, program code and other methods must be appropriately cited. Such materials are recognized as original intellectual contributions and afforded recognition through citation. a. All data sets and program code used in a publication must be cited in the text and listed in the reference section References for data sets and program code must include a persistent identifier, such as a Digital Object Identifier (DOI). b. Persistent identifiers ensure future access to unique published digital objects, such as a text or data set. Persistent identifiers are assigned to data sets by digital archives, such as institutional repositories and partners in the Data Preservation Alliance for the Social Sciences (Data-PASS). c. Data set citation example: Campbell, Angus, and Robert L. Kahn. American National Election Study, 1948. ICPSR07218-v3. Ann Arbor, MI: Inter-university Consortium for Political and Social Research [distributor], 1999. http://doi.org/10.3886/ICPSR07218.v3 *Level 3* All data, program code and other methods must be appropriately cited. Such materials are recognized as original intellectual contributions and afforded recognition through citation. Articles will not be published until the citations conform to these standards. a. All data sets and program code used in a publication must be cited in the text and listed in the reference section b. References for data sets and program code must include a persistent identifier, such as a Digital Object Identifier (DOI). Persistent identifiers ensure future access to unique published digital objects, such as a text or data set. Persistent identifiers are assigned to data sets by digital archives, such as institutional repositories and partners in the Data Preservation Alliance for the Social Sciences (Data-PASS). c. Data set citation example: Campbell, Angus, and Robert L. Kahn. American National Election Study, 1948. ICPSR07218-v3. Ann Arbor, MI: Inter-university Consortium for Political and Social Research [distributor], 1999. http://doi.org/10.3886/ICPSR07218.v3 **2, 3, 4. Data, Analytic Methods (Code), and Research Materials Transparency** *Level 1* The policy of the ___________ is to publish papers where authors indicate whether the data, methods used in the analysis, and materials used to conduct the research will be made available to any researcher for purposes of reproducing the results or replicating the procedure. 1. Authors must, in acknowledgments or the first footnote, indicate if they will or will not make their data, analytic methods, and study materials available to other researchers. 2. If an author agrees to make materials available, the author must specify where that material will be available. *Level 2* The policy of the ___________ is to publish papers only if the data, methods used in the analysis, and materials used to conduct the research are clearly and precisely documented and are maximally available to any researcher for purposes of reproducing the results or replicating the procedure. 1. Authors reusing data available from public repositories must provide program code, scripts for statistical packages, and other documentation sufficient to allow an informed researcher to precisely reproduce all published results. 2. Authors using original data must a. make the data available at a trusted digital repository (Note: If all data required to reproduce the reported analyses appears in the article text, tables, and figures then it does not also need to be posted to a repository.) b. include all variables, treatment conditions, and observations described in the manuscript c. provide a full account of the procedures used to collect, preprocess, clean, or generate the data d. provide program code, scripts, codebooks, and other documentation sufficient to precisely reproduce all published results e. provide research materials and description of procedures necessary to conduct an independent replication of the research. 3. In rare cases, despite authors’ best efforts, some or all data or materials cannot be shared for legal or ethical reasons. In such cases, authors must inform the editors at the time of submission. This will be taken into account during the review process. Authors are encouraged to anticipate data and material sharing at the beginning of their projects to provide for these circumstances. It is understood that in some cases access will be provided under restrictions to protect confidential or proprietary information. Editors may grant exceptions to data and material access requirements provided authors: a. explain the restrictions on the dataset or materials and how they preclude public access. b. provide a public description of the steps others should follow to request access to the data or materials. c. provide software and other documentation that will precisely reproduce all published results. d. provide access to all data and materials for which the constraints do not apply. 4. Data, program code, research materials, and other documentation of the research process should be made available through a trusted digital repository. Trusted repositories adhere to policies that make data discoverable, accessible, usable, and preserved for the long term. Trusted repositories also assign unique and persistent identifiers. For example these services are offered by partners in the Data Preservation Alliance for the Social Sciences (Data-PASS) and most institutional repositories. Author maintained websites are not compliant with this requirement. a. Dissemination of these materials may be delayed until publication. Under exceptional circumstances, editors may grant an embargo of the public release of data for at most one year after publication. b. Articles accepted for publication will not be assigned a publication date until the above conditions have been met. Authors are responsible for ensuring that their articles continue to meet these conditions. Failure to do so may lead to an editorial expression of concern or retraction of the article. *Level 3* The policy of the ___________ is to publish papers only if the data, methods used in the analysis, and materials used to conduct the research are clearly and precisely documented and are maximally available to any researcher for purposes of reproducing the results or replicating the procedure. All materials supporting the claims made by the author must be made available to the journal prior to publication. The journal, or an entity acting on behalf of the journal, will verify that the findings are replicable using the author’s data and methods of analysis. Failure to replicate at this stage may result in the paper not being published. 1. Authors reusing data available from public repositories must provide program code, scripts for statistical packages, and other documentation sufficient to allow an informed researcher to precisely reproduce all published results. 2. Authors using original data must a. make the data available at a trusted digital repository (Note: If all data required to reproduce the reported analyses appears in the article text, tables, and figures then it does not also need to be posted to a repository.) b. include all variables, treatment conditions, and observations described in the manuscript c. provide a full account of the procedures used to collect, preprocess, clean, or generate the data d. provide program code, scripts, codebooks, and other documentation sufficient to precisely reproduce all published results. e. provide research materials and description of procedures necessary to conduct an independent replication of the research 3. In rare cases, despite authors’ best efforts, some or all data or materials cannot be shared for legal or ethical reasons. In such cases, authors must inform the editors at the time of submission. This will be taken into account during the review process. Authors are encouraged to anticipate data and material sharing at the beginning of their projects to provide for these circumstances. It is understood that in some cases access will be provided under restrictions to protect confidential or proprietary information. Editors may grant exceptions to data and material access requirements provided authors: a. explain the restrictions on the dataset or materials and how they preclude public access. b. provide a public description of the steps others should follow to request access to the data or materials. c. provide software and other documentation that will precisely reproduce all published results. d. provide access to all data and materials for which the constraints do not apply. 4. Data, program code, research materials, and other documentation of the research process should be made available through a trusted digital repository. Trusted repositories adhere to policies that make data discoverable, accessible, usable, and preserved for the long term. Trusted repositories also assign unique and persistent identifiers. For example these services are offered by partners in the Data Preservation Alliance for the Social Sciences (Data-PASS) and most institutional repositories. Author maintained websites are not compliant with this requirement. a. Dissemination of these materials may be delayed until publication. Under exceptional circumstances, editors may grant an embargo of the public release of data for at most one year after publication. b. Articles accepted for publication will not be assigned a publication date until the above conditions have been met. Authors are responsible for ensuring that their articles continue to meet these conditions. Failure to do so may lead to an editorial expression of concern or retraction of the article. **5. Design and Analysis Transparency** *Level 1* The policy of the ___________ is to publish papers where authors follow standards for disclosing key aspects of the research design and data analysis. Authors are encouraged to review the standards available for many research applications from http://www.equator-network.org/ and use those that are relevant for the reported research applications. *Level 2* The policy of the ___________ is to publish papers where authors follow standards for disclosing key aspects of the research design and data analysis. Authors are required to review the standards available for many research applications from http://www.equator-network.org/ and use those that are relevant for the reported research applications. At manuscript submission, authors must confirm that they reviewed the standards, report whether any standards were relevant for the research application, and confirm that they followed those standards in the manuscript. *Level 3* The policy of the ___________ is to publish papers where authors follow standards for disclosing key aspects of the research design and data analysis. Authors are required to review the standards available for many research applications from http://www.equator-network.org/ and use those that are relevant for the reported research applications. At manuscript submission, authors must confirm that they reviewed the standards, report whether any standards were relevant for the research application, and confirm that they followed those standards in the manuscript. The journal, or an entity acting on behalf of the journal, will verify that the appropriate standards were adopted and followed. Failure to follow the relevant standards may result in the paper not being published. **6. Preregistration of Studies** *Level 1* The policy of the ___________ is to publish papers where authors indicate whether the conducted research was preregistered in an independent, institutional registry (e.g., http://clinicaltrials.gov/, http://socialscienceregistry.org/, http://osf.io/, http://egap.org/design-registration/, http://ridie.3ieimpact.org/). Preregistration of studies involves registering the study design, variables, and treatment conditions prior to conducting the research. 1. Authors must, in acknowledgments or the first footnote, indicate if they did or did not preregister the research in an independent, institutional registry. 2. If an author did preregister the research, the author must confirm that the study was registered prior to conducting the research with links to the time-stamped preregistrations at the institutional registry, and that the preregistration adheres to the disclosure requirements of the institutional registry or those required for the [preregistered badge maintained by the Center for Open Science][14]. *Level 2* The policy of the ___________ is to publish papers where authors indicate whether the conducted research was preregistered in an independent, institutional registry (e.g., http://clinicaltrials.gov/, http://socialscienceregistry.org/, http://osf.io/, http://egap.org/design-registration/, http://ridie.3ieimpact.org/). Preregistration of studies involves registering the study design, variables, and treatment conditions prior to conducting the research. A link to the preregistration in an institutional registry must be made available to the journal prior to publication. The journal, or an entity acting on behalf of the journal, will verify that preregistration adheres to the specifications for preregistration and then provide [certification of the preregistration][15] in the article. 1. Authors must, in acknowledgments or the first footnote, indicate if they did or did not preregister the research in an independent, institutional registry. 2. If an author did preregister the research, the author must confirm that the study was registered prior to conducting the research with links to the time-stamped preregistrations at the institutional registry, and that the preregistration adheres to the disclosure requirements of the institutional registry or those required for the [preregistered badge maintained by the Center for Open Science][16]. *Level 3* The policy of the ___________ is to publish papers only if the conducted research was preregistered in an independent, institutional registry (e.g., http://clinicaltrials.gov/, http://socialscienceregistry.org/, http://osf.io/, http://egap.org/design-registration/, http://ridie.3ieimpact.org/). Preregistration of studies involves registering the study design, variables, and treatment conditions prior to conducting the research. Link to the preregistration in an institutional registry must be made available to the journal prior to publication. The journal, or an entity acting on behalf of the journal, will verify that preregistration adheres to the specifications for preregistration and then provide [certification of the preregistration][17] in the article. Failure to adhere to preregistration requirements may result in the paper not being published. 1. Authors must, in acknowledgments or the first footnote, indicate if they did or did not preregister the research in an independent, institutional registry. 2. If an author did preregister the research, the author must confirm that the study was registered prior to conducting the research with links to the time-stamped preregistrations at the institutional registry, and that the preregistration adheres to the disclosure requirements of the institutional registry or those required for the [preregistered badge maintained by the Center for Open Science][18]. **7. Preregistration of Analysis Plans** *Level 1* The policy of the ___________ is to publish papers where authors indicate whether or not the conducted research was preregistered with an analysis plan in an independent, institutional registry (e.g., http://clinicaltrials.gov/, http://socialscienceregistry.org/, http://osf.io/, http://egap.org/design-registration/, http://ridie.3ieimpact.org/). Preregistration of studies involves registering the study design, variables, and treatment conditions. Including an analysis plan involves specification of sequence of analyses or the statistical model that will be reported. 1. Authors must, in acknowledgments or the first footnote, indicate if they did or did not preregister the research with or without an analysis plan in an independent, institutional registry. 2. If an author did preregister the research with an analysis plan, the author must: a. confirm in the text that the study was registered prior to conducting the research with links to the time-stamped preregistrations at the institutional registry, and that the preregistration adheres to the disclosure requirements of the institutional registry or those required for the [preregistered badge with analysis plans maintained by the Center for Open Science][19]. b. Report all pre-registered analyses in the text, or, if there were changes in the analysis plan following preregistration, those changes must be disclosed with explanation for the changes. c. Clearly distinguish in text analyses that were preregistered from those that were not, such as having separate sections in the results for confirmatory and exploratory analyses. *Level 2* The policy of the ___________ is to publish papers where authors indicate whether or not the conducted research was preregistered with an analysis plan in an independent, institutional registry (e.g., http://clinicaltrials.gov/, http://socialscienceregistry.org/, http://osf.io/, http://egap.org/design-registration/, http://ridie.3ieimpact.org/). Preregistration of studies involves registering the study design, variables, and treatment conditions. Including an analysis plan involves specification of sequence of analyses or the statistical model that will be reported. A link to the preregistration in an institutional registry must be made available to the journal prior to publication. The journal, or an entity acting on behalf of the journal, will verify that preregistration adheres to the specifications for preregistration and then provide [certification of the preregistration][20] in the article. 1. Authors must, in acknowledgments or the first footnote, indicate if they did or did not preregister the research with or without an analysis plan in an independent, institutional registry. 2. If an author did preregister the research, the author must: a. confirm that the study was registered prior to conducting the research with links to the time-stamped preregistrations at the institutional registry, and that the preregistration adheres to the disclosure requirements of the institutional registry or those required for the [preregistered badge with analysis plans maintained by the Center for Open Science][21]. b. Report all pre-registered analyses in the text, or, if there were changes in the analysis plan following preregistration, those changes must be disclosed with explanation for the changes. c. Clearly distinguish in text analyses that were preregistered from those that were not, such as having separate sections in the results for confirmatory and exploratory analyses. *Level 3* The policy of the ___________ is to publish papers only if the conducted research was preregistered with an analysis plan in an independent, institutional registry (e.g., http://clinicaltrials.gov/, http://socialscienceregistry.org/, http://osf.io/, http://egap.org/design-registration/, http://ridie.3ieimpact.org/). Preregistration of studies involves registering the study design, variables, and treatment conditions. Including an analysis plan involves specification of sequence of analyses or the statistical model that will be reported. A link to the preregistration in an institutional registry must be made available to the journal prior to publication. The journal, or an entity acting on behalf of the journal, will verify that preregistration adheres to the specifications for preregistration and then provide [certification of the preregistration][22] in the article. 1. Authors must, in acknowledgments or the first footnote, indicate if they did or did not preregister the research with or without an analysis plan in an independent, institutional registry. 2. If an author did preregister the research, the author must: a. confirm that the study was registered prior to conducting the research with links to the time-stamped preregistrations at the institutional registry, and that the preregistration adheres to the disclosure requirements of the institutional registry or those required for the [preregistered badge with analysis plans maintained by the Center for Open Science][23]. b. Report all pre-registered analyses in the text, or, if there were changes in the analysis plan following preregistration, those changes must be disclosed with explanation for the changes. c. Clearly distinguish in text analyses that were preregistered from those that were not, such as having separate sections in the results for confirmatory and exploratory analyses. **8. Replication** *Level 1* The policy of the ___________ is to encourage submission of replication studies, particularly of research published in this journal. *Note: If the journal will publish replication studies in a separate section, or as online-only in case of print journals, that should be made explicit.* *Level 2* The policy of the ___________ is to encourage submission of replication studies, particularly of research published in this journal. Replication studies are normally reviewed in two stages, the first being results-blind. 1. On initial submission, authors of replication studies should: a. Denote in the cover letter that the manuscript is a replication study submission b. Include a full manuscript for the abstract, introduction, and methods without the results and discussion sections. The submitted manuscript must not indicate information about the outcome-relevant results. c. The methods must contain a complete analysis plan of what is to be included in the full article. d. If relevant, outcome-irrelevant results can be reported to demonstrate, for example, that that experimental manipulations were effective, or outcome variables were measured reliably and conformed to distributional assumptions. 2. If the submission passes initial review, then the authors will submit a complete manuscript for second stage review to confirm that the final report adequately addresses reviewer concerns from the initial submission. *Level 3* The policy of the ___________ is to encourage submission of replication studies, particularly of research published in this journal. When possible, replication studies are reviewed in two stages following the [Registered Reports][24] format. In particular, the first stage of review is conducted prior to the data being collected or, for existing datasets, before the outcomes are observed: 1. On initial submission, authors should: a. Denote in the cover letter that the manuscript is Registered Report submission and confirm that the data do not exist, or that the outcomes have not been observed b. Include a full manuscript for the abstract, introduction, and methods without the results and discussion sections. c. The methods must contain a complete analysis plan of what is to be included in the full article. 2. If the submission passes initial review, then the authors will receive an in-principle acceptance prior to data collection or analysis of the outcomes. 3. For the second stage of review, authors submit a complete manuscript. Reviewers assess the extent to which the authors followed the preregistered design and/or analysis plan, and evaluate non-outcome relevant criteria (e.g., manipulation checks) to confirm whether the research was an effective test of the research question. [1]: http://cos.io/ [2]: http://cos.io/top [3]: https://osf.io/2cz65/ [4]: https://docs.google.com/document/d/1JvswEe5X0aCY02zyCoyiFbdKlij24xUnuO2Qa6HJiIo/edit [5]: http://www.psychologicalscience.org/publications/psychological_science/ps-submissions#DISC [6]: https://osf.io/tvyxz/wiki/home/ [7]: https://osf.io/tvyxz/wiki/home/ [8]: https://osf.io/8mpji/wiki/home/ [9]: https://osf.io/tvyxz/wiki/home/ [10]: https://osf.io/tvyxz/wiki/home/ [11]: https://osf.io/8mpji/wiki/home/ [12]: https://osf.io/8mpji/wiki/home/ [13]: https://osf.io/8mpji/wiki/home/ [14]: https://osf.io/tvyxz/wiki/home/ [15]: https://osf.io/tvyxz/wiki/home/ [16]: https://osf.io/tvyxz/wiki/home/ [17]: https://osf.io/tvyxz/wiki/home/ [18]: https://osf.io/tvyxz/wiki/home/ [19]: https://osf.io/tvyxz/wiki/home/ [20]: https://osf.io/tvyxz/wiki/home/ [21]: https://osf.io/tvyxz/wiki/home/ [22]: https://osf.io/tvyxz/wiki/home/ [23]: https://osf.io/tvyxz/wiki/home/ [24]: https://osf.io/8mpji/wiki/home/