**Title:** **Research Reproducibility in Theory and Practice (Examples and Focus on Biological Sciences)** **Instructors:** - Anita Bandrowski, PhD, SciCrunch and University of California, San Diego; https://orcid.org/0000-0002-5497-0243 - Daniel S. Katz, PhD, Assistant Director for Scientific Software and Applications, National Center for Supercomputing Applications and Research; and Research Associate Professor in Information Sciences, Computer Science, and Electrical and Computer Engineering, University of Illinois Urbana-Champaign; http://danielskatz.org/ - Tracey Weissgerber, PhD, Mayo Clinic and QUEST (Quality, Ethics, Open Science, Translation), Charité Universitätsmedizin Berlin, Germany; [My Bibliography][1] **Description:** This course will focus on issues of reproducibility in research from a broad perspective. It will include an introduction to the differing types of reproducibility, and a discussion of new grant review guidelines and the philosophy that underpins them. The course will look at reproducibility in several contexts, including collecting and communication in experimental research, providing a robust record of computational research, and the limitations and debates around these approaches. We will introduce several tools and approaches to support reproducible research practice, including the RRID portal, Zenodo, Jupyter Notebooks, and best practice in research and data management, communication, and open sharing. There will be three 1.5-hour sessions offering a mix of lecture and practical work, particularly information gathering and analysis. The emphasis will be on providing frameworks within which information can be gathered and understood rather than on “fact teaching.” **Audience level:** Beginner **Intended audience:** The target audience is researchers seeking a deeper understanding of reproducibility in a variety of contexts, as well as those with a need to support researchers – for example, staff from research offices, libraries, service providers, or publishers. Participants should be seeking an introduction to working toward reproducibility in practice and to the tools that can support them in doing this. **Requirements:** Participants should have basic computer skills and a laptop. **Course Learning Objectives** - At the end of this course, participants will understand: - The difference between replication, repeatability, and reproducibility. - How to submit protocols to protocols.io and know when it is appropriate to do so. - How to submit code or data to appropriate repositories and know when it is appropriate. - How to find RRIDs for tools used in a study and know when it is appropriate. - How to create a successful Key Biological Resource authentication plan (for NIH grants). - How to create a simple Jupyter Notebook with both data and code. - Reproducibility challenges in data science and computational science. **Course Topics** This course will be presented over four days for 1.5 hours each day and will cover the following topics: - Protocols.io will be presented - Key Biological and Chemical Resources and what does authentication mean? Students should be able to describe what the difference is between authentication of reagents and identification of reagents. What are the best practices for antibodies, organisms, cell lines and of course chemicals? What does authentication mean and how does one create a reasonable authentication document for Key Biological and Chemical Resources for your next grant? - Code reproducibility is a big problem, and in theory one that should be easiest to solve. We will explore some of the issues that make code not reproducible and help students improve practices to make their code more transparent and reusable. - Jupyter Notebooks will be presented as a tool that students should become familiar with, and then they will be used by several exercises in later lectures. **Course Schedule** **Day 1: Wednesday** - Projects/Resources that are addressing aspects of reproducibility, how can we make sense of all this noise? (Bandrowski) - Reproducibility exercise using your favorite tools (Bandrowski) - Reagents and RRIDs, how can we ensure that reagents are properly identified (Bandrowski) - Authentication of research resources, when is this critical - How to create your successful Key Biological Resources Authentication document Slides for Day 1: https://docs.google.com/presentation/d/1Fq5_taje-t842oYVEJWINCtOYA_C8gidVsXAyCC2OH8/edit?usp=sharing **Day 2: Monday** - Protocols.io: Walking through a protocol and creating a new one - Tools and examples for visualization. Are we making errors with how we represent data? **Day 3: Wednesday** - Opening exercise - What is reproducibility? - What is data science, and what is computational science? - Computational reproducibility principles 1. Provide structure 2. Control the source & changes 3. Use notebooks to explain and document 4. Automate steps 5. Automate everything 6. Capture the environment 7. Provide a license & make citable - Closing exercise Slides for Day 3: [ppt](https://github.com/danielskatz/repro-fdtd1d/blob/master/Computational_Reproducibility.pptx), [pdf](https://github.com/danielskatz/repro-fdtd1d/blob/master/Computational_Reproducibility.pdf) **Day 4: Monday: TBD** **Course Materials and Supplies Required** Students are expected to have the following material before the course begins. - Own laptop - Internet access - Hypothes.is account - Google mail account **Other Resources** [1]: https://www.ncbi.nlm.nih.gov/sites/myncbi/tracey.weissgerber.1/bibliography/45097548/public/?sort=date&direction=descending