# [Enabling Transparency in Special Education Research][1] # ---------- [Agenda and Working Groups][2] ([PDF][3]) List of working groups [members][4] ## Notes and Working Docs for Each Group ## - **[Data, Materials, and Code Transparency][5]** - osf.io/ns2jh/ - **[Bias Reduction Working Group][6]** - osf.io/zn8fk/ - **[Reporting Guidelines Working Group][7]** - osf.io/8zhbc/ - **[Transparency in Peer Review, Preprints][8]** - osf.io/xnjp6/ - **[Incentives for Transparency Working Group][9]** - osf.io/sqebr/ ## Outputs for this meeting ## This [document][10] will contain the output materials for this meeting. ### Some informal data relevant to this meeting ### **[Summary data from the opinion survey][11]** (N = 26 meeting attendees) **[TOP policy evaluation for Special Education journals][12]** (N = 20 journals participating in this meeting) ## Background Reading and Relevant Research ## **Readings** - [Promoting Open Science to Increase the Trustworthiness of Evidence in Special Education][13] - "...We propose that researchers in special education adopt emerging open-science reforms, such as preprints, data and materials sharing, preregistration of studies and analysis plans, and Registered Reports. Adoption of these practices will require shifts in cultural norms, guidelines, and incentives. We discuss how adopting open-science practices can advance the quality of research and, consequently, policy and practice in special education." - [Promoting an open research culture][14] - Covers the rationale and background for the TOP Guidelines. More details about TOP are here ([cos.io/top][15]), including a [summary table][16] and [complete TOP policy language][17]. - The [Preregistration Revolution][18] - Covers the benefits of preregistration for most hypothesis testing research and situations in which preregistration is beneficial but not typical. - [A recipe for rigor][19] - "A simple strategy to avoid bias—declaring in advance what you will study, and how—is rapidly catching on." **Research** - [Badges to Acknowledge Open Practices: A Simple, Low-Cost, Effective Method for Increasing Transparency][20] - Issuing badges to acknowledge open data is associated with an increased rate of data sharing, without issuing any mandate from the journal. - ![badge_figure][21] - [Many Analysts, One Data Set: Making Transparent How Variations in Analytic Choices Affect Results][22] - Different teams of data analysts will come up with different analytical strategies, and conclusions, even when using the same dataset and research questions - ![many_analysts][23] - [Estimating the reproducibility of psychological science][24] ([OA][25]) - When attempting to replicate previously published research using high powered, preregistered designs, most results cannot be reproduced. - ![RPP][26] [1]: https://osf.io/s4z7n/wiki/home/? [2]: https://docs.google.com/document/d/1dDsQqNxWglCfa5ltW_16Rn-EALsVM6H5zNCQwRT-2FE/edit# [3]: https://osf.io/2dpgq/ [4]: https://docs.google.com/spreadsheets/d/1uy4Hnw2tMOs9tHDAY-wHlxACQpdZWuWbg78yoH0gqb8/edit#gid=0 [5]: https://docs.google.com/document/d/16X2MRL9RmaUQeW0hn2JMqy-JSxPW8wvFaPyRPpzcRSM/edit#heading=h.t7u0fcxin753 [6]: https://docs.google.com/document/d/1Xt7ELBZkE1KHp9Gdvi_hWzjGWUvuVxo8-izhC4tU0EI/edit [7]: https://docs.google.com/document/d/18T9vpA-bOGLNhT57OL4EJHUvmZqRUymtaXRCQy3snh8/edit#heading=h.25lka1dzi5g [8]: https://docs.google.com/document/d/11p4DoL7zettZaBDSJShjlJcZKwXI2Kjk7tvvdX9hlaQ/edit# [9]: https://docs.google.com/document/d/1QSr5ZEwd32BhOijJgkwc8ua-qubehmY-3_VjfEG8FbM/edit# [10]: https://docs.google.com/document/d/1FG0FXLMsHtiteUeMfCimZiRqbj8Ci9ZaMU4NU4phyb8/edit# [11]: https://osf.io/q5vt3/ [12]: https://osf.io/twndy/ [13]: http://journals.sagepub.com/doi/full/10.1177/0014402918793138 [14]: http://science.sciencemag.org/content/348/6242/1422 [15]: http://cos.io/top [16]: https://osf.io/2cz65/ [17]: https://osf.io/9f6gx/wiki/Guidelines/ [18]: http://www.pnas.org/content/early/2018/03/08/1708274114 [19]: http://science.sciencemag.org/content/361/6408/1192 [20]: https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1002456 [21]: https://cdn.cos.io/media/images/badges_chart.original.png [22]: http://journals.sagepub.com/doi/10.1177/2515245917747646 [23]: https://fivethirtyeight.com/wp-content/uploads/2015/08/truth-vigilantes-soccer-calls2.png [24]: http://science.sciencemag.org/content/349/6251/aac4716 [25]: http://eprints.keele.ac.uk/877/1/Open%20Science%20%28Science%20Pre-Print%29.pdf [26]: https://files.osf.io/v1/resources/s4z7n/providers/osfstorage/5baa301011fc4b00191cdad3?mode=render