## **Code peer review**: towards more reproducible and reliable research ----- This workshop is offered as part of the [**Open Science Community - Wageningen**](https://openscience-wageningen.com/) lunch time seminar series. You can find more information about upcoming OSC-W events [here](https://openscience-wageningen.com/events/). **Organisers**: - Stefan Vriend, LTER-LIFE Coordinator Veluwe (s.vriend@nioo.knaw.nl) - Joey Burant, Tenure-track Researcher (j.burant@nioo.knaw.nl) **Date**: 23 November 2023 @ 12:00 - 13:00 **Location**: Colloquium Room (1.2.01), Netherlands Institute of Ecology (NIOO-KNAW), Droevendaalsesteeg 10, Wageningen *If you are visiting NIOO, please check in with the reception upon arrival. ----- ### Schedule **12:00 - 12:12** The basics of code peer review **12:12 - 12:15** Introduction to the interactive session **12:15 - 12:50** Break-out groups/pairs — applying the code review checklist to real world examples **12:50 - 13:00** Discussion and concluding remarks ----- ### Workshop materials - Introductory [**slides**](https://osf.io/duyhb) - Google form with the [**14-step code review checklist**](https://forms.gle/zEpCtG7fuaB62P1W9) - Many of the concepts presented in this workshop are formalised in a recent paper from members of the [Society for Open, Reliable, and Transparent Ecology and Evolutionary biology](https://sortee.org/): Ivimey-Cook ER, Pick JL, Bairos-Novak KR, Culina A, Gould E, Grainger M, Marshall BM, Moreau D, Paquet M, Royauté R, Sánchez-Tójar A, Silva I, Windecker SM (2023) **Implementing code review in the scientific workflow: insights from ecology and evolutionary biology**. *Journal of Evolutionary Biology*, 36: 1347-1356. https://doi.org/10.1111/jeb.14230 [**PDF**](https://osf.io/sd9je) #### Options for code review activity Select **one** of the following publications and download the corresponding data and R code. For the purposes of the peer review activity, you will want to open and run through the code in R/RStudio (or your preferred coding environment). - **Option 1**: Woodman et al. (2022), *Journal of Animal Ecology* - [paper](https://doi.org/10.1111/1365-2656.13851) - [data and readme on Dryad](https://doi.org/10.5061/dryad.m63xsj44s) - [code on Zenodo](https://doi.org/10.5281/zenodo.7327335) # - **Option 2**: Gonzalez et al. (2022), *Ecology* - [paper](https://doi.org/10.1002/ecy.4120) - [data and readme Dryad](https://doi.org/10.5061/dryad.2547d7wwk) - [code on Zenodo](https://doi.org/10.5281/zenodo.7916802) # - **Option 3**: Pollierer et al. (2023), *Ecology Letters* - [paper](https://doi.org/10.1111/ele.14276) - [data and readme on Dryad](https://doi.org/10.5061/dryad.2280gb5xv) - [code on Zenodo](https://doi.org/10.5281/zenodo.7993642) (note: there are multiple scripts available; focus on the main script "Pollierer_FoodWebReconstruction.Rmd") # - **Option 4**: Govaert et al. (2023), *Ecology* - [paper](https://doi.org/10.1002/ecy.4199) - [data and readme on Dryad](https://datadryad.org/stash/dataset/doi:10.5061/dryad.r4xgxd2jf) - [code on Zenodo](https://zenodo.org/records/7827593) # ----- ### 14-Step Checklist For the interactive session, use the link above. 1. Does the code match the methods as described in the corresponding article, metadata, and/or documentation? 2. Does the code run? 3. Is the code reliable? 4. Does the code include "unit tests" or other checks that verify the code is working as intended? For example, following a bit of data wrangling or transformation, is there code that checks whether the transformation has been accurately implemented? 5. Are the results and conclusion reproducible from the code as provided? 6. Does the code specify which versions of packages/libraries were used? 7. Does the code include clear documentation that detail the rationale behind, e.g., data wrangling decisions and analytical approaches? 8. Is the whole workflow code/script-based? Is it self contained (e.g., the code does not involve steps outside the script or pipeline, such as manual manipulation in other software like Excel)? 9. Is the code readable? For example, is the code annotated? Does the code apply appropriate indentation? Does the code include headers and sections? 10. Is the code consistently formatted? 11. Does the code use informative names for objects? 12. Is the code efficient? For example, does it implement functional programming (e.g., does it use generalised, custom functions or for loops to repeat processes)? 13. Does the code avoid hard-coding (i.e., does the code include sections that assign fixed values or data directly rather than using variables)? 14. Does the code explicitly calls the package namespace when calling functions (i.e., `package::function()`)? ----- ### Additional Resources and Readings - BES (2017) **A guide to reproducible code in ecology and evolution**. *British Ecological Society*. https://www.britishecologicalsociety.org/wp-content/uploads/2019/06/BES-Guide-Reproducible-Code-2019.pdf [**PDF**](https://osf.io/8gryd) - Culina A, van den Berg I, Evans S, Sánchez-Tójar A (2020) **Low availability of code in ecology: a call for urgent action**. *PLoS Biology*, 18: e3000763. https://doi.org/10.1371/journal.pbio.3000763 [**PDF**](https://osf.io/p9qty) - O'Dea RE, Parker TH, Chee YE, Culina A, Drobniak SM, Dunacan DH, Fidler F, Gould E, Ihle M, Kelly CD, Lagisz M, Roche DG, Sánchez-Tójar A, Wilkinson DP, Wintle BC, Nakagawa S (2021) **Towards open, reliable, and transparent ecology and evolutionary biology**. *BMC Biology*, 19: 68. https://doi.org/10.1186/s12915-021-01006-3 [**PDF**](https://osf.io/973j2) - Filazzola A, Lortie CJ (2022) **A call for clean code to effectively communicate science**. *Methods in Ecology and Evolution*, 13: 2119-2128. https://doi.org/10.1111/2041-210X.13961 [**PDF**](https://osf.io/7ca26) - Jenkins GB, Beckerman AP, Bellward C, Benítez-López A, Ellison AM, Foote CG, Hufton AL, Lashley MA, Lortie CJ, Ma Z, Moore AJ, Narum SR, Nilsson J, O'Boyle B, Provete DB, Razgour O, Rieseberg L, Riginos C, Santini L, Sibbett B, Peres-Neto PR (2023) **Reproduciblity in ecology and evolution: minimum standards for data and code**. *Ecology and Evolution*, 13: e9961. https://doi.org/10.1002/ece3.9961 [**PDF**](https://osf.io/q8ygv)