## **Code peer review**: towards more reproducible and reliable research ----- This workshop is offered as part of the [SORTEE Code Club](https://www.sortee.org/blog/2024/03/06/2024_code_club_kickoff/). **Organisers**: - Stefan Vriend, LTER-LIFE Coordinator Veluwe (s.vriend@nioo.knaw.nl) - Joey Burant, Tenure-track Researcher (j.burant@nioo.knaw.nl) - Freddy Hillemann, SPI-Birds Data Science Technician (f.hillemann@nioo.knaw.nl) **Date**: 16 April 2024 @ 10:00 - 11:00 CEST (UTC + 2h) **Location**: SORTEE Code Club @ Zoom ----- ### Schedule **10:00 - 10:15** Introduction: why and how to do code peer review **10:15 - 10:45** Practical session: demonstration of code review **10:45 - 11:00** Discussion and concluding remarks ----- ### Workshop materials - Introductory [**slides**](https://osf.io/tpdjf) - Google form with the [**17-step code review checklist**](https://forms.gle/7bmDwTg7DiFGZqPt5) - Many of the concepts presented in this workshop are formalised in a recent paper from members of the [Society for Open, Reliable, and Transparent Ecology and Evolutionary biology](https://sortee.org/): Ivimey-Cook ER, Pick JL, Bairos-Novak KR, Culina A, Gould E, Grainger M, Marshall BM, Moreau D, Paquet M, Royauté R, Sánchez-Tójar A, Silva I, Windecker SM (2023) **Implementing code review in the scientific workflow: insights from ecology and evolutionary biology**. *Journal of Evolutionary Biology*, 36: 1347-1356. https://doi.org/10.1111/jeb.14230 [**PDF**](https://osf.io/sd9je) #### Use case for code review activity We will use a paper by Gonzalez et al., published in 2023 in Ecology, to demonstrate the code review process. To join the practical session, download the data and R code below. For the purposes of the peer review activity, you will want to open and run through the code in R/RStudio (or your preferred coding environment). - Gonzalez AD, Pearse IS, Redmond MD (2023), **Increased aridity is associated with stronger tradeoffs in ponderosa pine vital functions**. *Ecology*, 104: e4120. - [paper](https://doi.org/10.1002/ecy.4120)( [PDF](https://osf.io/v36wt)) - [data and readme on Dryad](https://doi.org/10.5061/dryad.2547d7wwk) - [code on Zenodo](https://doi.org/10.5281/zenodo.7916802) # ----- ### 17-Step Checklist For the interactive session, use the link above. 1. Does the code match the methods as described in the corresponding article, metadata, and/or documentation? 2. Does the code run in its entirety and without issue? 3. Is the code reliable? 4. Does the code include "unit tests" or other checks that verify the code is working as intended? For example, following a bit of data wrangling or transformation, is there code that checks whether the transformation has been accurately implemented? 5. Are the results and conclusions reproducible from the code as provided? 6. Does the code specify which versions of packages/libraries were used? 7. Does the code include clear documentation that detail the rationale behind, e.g., data wrangling decisions and analytical approaches? 8. Is the whole workflow code/script-based? Is it self contained (e.g., the code does not involve steps outside the script or pipeline, such as manual manipulation in other software like Excel)? 9. Is the code readable? For example, is the code annotated? Does the code apply appropriate indentation? Does the code include headers and sections? 10. Is the code consistently formatted? 11. Does the code use informative names for objects? 12. Is the code efficient? For example, does it implement functional programming (e.g., does it use generalised, custom functions or for loops to repeat processes)? 13. Does the code avoid hard-coding (i.e., does the code include sections that assign fixed values or data directly rather than using variables)? 14. Does the code explicitly calls the package namespace when calling functions (i.e., `package::function()`)? 15. Has a license (e.g., MIT, Apache License 2.0, or equivalent) been applied to the code? 16. Does the documentation (metadata, README) include clear citations for related materials like data and the corresponding article? This may not be applicable in all instances. 17. Does the code have a persistent identified (e.g., DOI)? ----- ### Additional Resources and Readings - BES (2017) **A guide to reproducible code in ecology and evolution**. *British Ecological Society*. https://www.britishecologicalsociety.org/wp-content/uploads/2019/06/BES-Guide-Reproducible-Code-2019.pdf [**PDF**](https://osf.io/8gryd) - Culina A, van den Berg I, Evans S, Sánchez-Tójar A (2020) **Low availability of code in ecology: a call for urgent action**. *PLoS Biology*, 18: e3000763. https://doi.org/10.1371/journal.pbio.3000763 [**PDF**](https://osf.io/p9qty) - O'Dea RE, Parker TH, Chee YE, Culina A, Drobniak SM, Dunacan DH, Fidler F, Gould E, Ihle M, Kelly CD, Lagisz M, Roche DG, Sánchez-Tójar A, Wilkinson DP, Wintle BC, Nakagawa S (2021) **Towards open, reliable, and transparent ecology and evolutionary biology**. *BMC Biology*, 19: 68. https://doi.org/10.1186/s12915-021-01006-3 [**PDF**](https://osf.io/973j2) - Filazzola A, Lortie CJ (2022) **A call for clean code to effectively communicate science**. *Methods in Ecology and Evolution*, 13: 2119-2128. https://doi.org/10.1111/2041-210X.13961 [**PDF**](https://osf.io/7ca26) - Jenkins GB, Beckerman AP, Bellward C, Benítez-López A, Ellison AM, Foote CG, Hufton AL, Lashley MA, Lortie CJ, Ma Z, Moore AJ, Narum SR, Nilsson J, O'Boyle B, Provete DB, Razgour O, Rieseberg L, Riginos C, Santini L, Sibbett B, Peres-Neto PR (2023) **Reproduciblity in ecology and evolution: minimum standards for data and code**. *Ecology and Evolution*, 13: e9961. https://doi.org/10.1002/ece3.9961 [**PDF**](https://osf.io/q8ygv)