### **Changelog:** - **2025-06-24:** The electronic supplemental materials have been updated in the OSF repository to ensure that the tuning syntax in the ESM chapter *Theory and Application of Nested Resampling* still works for current software versions of the mlr3verse. - The files *code/electronic_supplemental_material.Rmd*, *results/electronic_supplemental_material.pdf*, *results/sessioninfo_electronic_supplemental_material.txt* and all corresponding files in *results/electronic_supplemental_material/files* have been updated. - All remaining files (including *renv.lock*) remain unchanged and the old versions are still available using the versioning system of the OSF. - The Code Ocean repository remains unchanged and still contains the old syntax that only works for the old software versions at the time the paper was published. ---- This repository provides all materials (electronic supplemental materials, reproducible manuscript, practical exercises) for the tutorial paper: ### *"Best Practices in Supervised Machine Learning: A Tutorial for Psychologists"* Please cite as: >Pargent, F., Schoedel, R., & Stachl, C. (2023). Best Practices in Supervised Machine Learning: A Tutorial for Psychologists. Advances in Methods and Practices in Psychological Science, 6(3). https://doi.org/10.1177/25152459231162559 ---- #### If you are looking for the **Electronic Supplemental Materials (ESM)** that are referenced in the tutorial paper, please open the file **electronic_supplemental_material.pdf** in the results folder. ---- #### If you want to follow along with the exercises in the tutorial or the ESM without installing software on your own computer, we recommend our resources on **Code Ocean** at **<https://doi.org/10.24433/CO.5687964.v1>**. Go there for an instruction on how to run our exercises or the ESM directly in your browser. ---- The most important files in the OSF Storage folder are: - **code/manuscript.Rmd**: main Rmarkdown file of our manuscript that creates the PDF **manuscript.pdf** when knitted. - **code/exercises/**: folder containing the exercises included in the tutorial as stand-alone Rmarkdown files - **1_performance_evaluation.Rmd** - **2_train_random_forest.Rmd** - **3_benchmark_experiments.Rmd** - **4_interpretable_ml.Rmd** - **code/electronic_supplemental_material.Rmd**: Rmarkdown file of our ESM that creates the PDF **electronic_supplemental_material.pdf** when knitted - **data/clusterdata.RDS**: the PhoneStudy dataset used throughout the tutorial; this is a copy of the data published in https://osf.io/kqjhr/. - **mlr3TutorialPaper.Rproj**: RStudio project file; this provides a convenient way to download the necessary R package version together with the *renv* package (see instructions below) - **renv.lock**: lock file documenting the package versions used for our analyses. - **code/references.bib** and **code/r-references.bib**: Bibfiles with the complete literature and R packages referenced in the manuscript and in the ESM ---- How to practice with the exercises included in the tutorial **on your own computer**: - The tutorial paper includes a practical exercise for each module (Practical Exercises I to IV) that apply the newly introduced concepts with mlr3 in R. - These exercises are provided in the **code/exercises/** folder as stand-alone Rmarkdown files. - To follow along with an exercises without having to copy the code from the manuscript, you can open one of these exercises files in *RStudio* and run the code chunks in consecutive oder by clicking on the *play* buttons in *RStudio*. - Each exercise can be run separately. Exercises only depend on the data file **data/clusterdata.RDS** but not on the code in previous exercises. - Instead of installing all the requires R packages manually, we recommend to follow the steps in the next section, which includes instructions on how to set up the software on your computer so that you are ready to begin with the exercises. ---- To reproduce our manuscript or our electronic supplemental materials **on your own computer**, please follow these steps: - Make sure to have *R* and *RStudio* installed: - You can download *R* from https://cran.r-project.org/ - You can download *RStudio Desktop (free)* from https://posit.co/download/rstudio-desktop/ - Download the complete OSF Storage ("Download as zip"). Unpack the ZIP file on your computer and double-click the **mlr3TutorialPaper.Rproj** file. RStudio should start and automatically construct a project-specific *renv* directory for the packages used in this project. By typing `renv::restore()` into the *RStudio* console, R should automatically install and updates all required packages to the exact versions we used when constructing the tutorial. (If your R version is newer than the one reported by *renv*, this should usually not be a problem. Hopefully, newer R package versions will still work with our code, but there can be no guarantee...) - You should now be able to open one of the 4 exercise files included in the **code/exercises/** folder and run the code chunks in consecutive order by simply clicking on the *play* buttons in *RStudio*. - Similarly, you should also be able to run the code chunks in the Rmarkdown files **code/manuscript.Rmd** and **code/electronic_supplemental_material.Rmd**. - If you also want to build the PDFs **manuscript.pdf** and **electronic_supplemental_material.pdf** by yourself (in addition to the steps explained earlier) also you need a working *TeX* installation. - If you do not already have something like *TeX_Live*, *MacTeX* or *MiKTeX* installed on your computer, we recommend to install *TinyTeX* by running the following commands in RStudio: `tinytex::install_tinytex()` - When you now have Rmarkdown files open in RStudio and click on the *knit* button, this should reproduce the respective PDF. ---------- For questions or feedback, please contact the authors at **florian.pargent@psy.lmu.de** **ramona.schoedel@psy.lmu.de** **clemens.stachl@unisg.ch**