## **Seminar: Systematic Review and Meta-Analysis** Fernando Ferreira-Santos (FPCEUP) | http://ferreira-santos.eu ([Version 8][1]; Updated 2024-05-13) ---------- ### Seminar materials and evaluation This OSF project contains supporting materials for my Seminars on Systematic Review/Systematic Research Synthesis and Meta-analysis. Below you will find an overview of the seminar and relevant information on the topic, including **instructions for installing software for the seminar**. You may access the **seminar materials** and **evaluation** (when applicable) in the private "Files" section by following the link provided in class. The instructions and deadlines for the evaluation are indicated in the document itself. Read carefully. ![Read carefully GIF][2] ---------- ### Seminar overview **Title:** - Systematic review of the literature and meta-analysis in Psychology **Summary:** - In this seminar, we will discuss a) the typology of literature reviews and syntheses (e.g., systematic/non-systematic, narrative/quantitative) and the advantages of systematic reviews and meta-analyses; and b) the strength of empirical evidence as function of the quality of studies and respective methodological design. We will then focus on the statistical procedures for the estimation, meta-analytic combination, and analysis of the heterogeneity of effect sizes. These features will be complemented with practical exercises. **Core references:** - Cooper, H. (2017). *Research synthesis and meta-analysis: A step-by-step approach* (5th ed.). Thousand Oaks, CA: Sage. - Cooper, H., Hedges, L. V., & Valentine, J. C. (Eds.) (2019). *Handbook of research synthesis and meta-analysis* (3rd ed.). New York: Russell Sage Foundation ---------- ### Software for systematic reviews - **The Systematic Review Toolbox** (http://www.systematicreviewtools.com/index.php) - Search engine for systematic review tools! Includes logical tools/instruments (e.g., risk of bias scales) as well as software   - **Rayyan** (https://rayyan.ai) - Systematic review manager allowing collaborative screening of abstracts (free and paid tiers; web- and app-based)   - **Abstrackr** (http://abstrackr.cebm.brown.edu) - Abstract screening assistant (free; web-based but non-https website)   - **ASReview** (https://asreview.nl/) - Text screening assistant that uses active (machine) learning to reorder documents (up-voting those likely to be more relevant) based on your previous screening decisions (free, open-source; local installation)   - **RevMan Web** (https://revman.cochrane.org) - Review manager for Cochrane reviews (free; web-based)   - **Covidence** (https://www.covidence.org/) - Systematic review manager (commercial; web-based)   - **Resources for Conducting Systematic Reviews (Center on KTDRR)** (https://ktdrr.org/resources/sr-resources/index.html)) - Overview of review steps and helpful tools, including software ---------- ### Free software for meta-analysis - **JASP** - JASP (once thought to mean "Just Another Statistical Package", but cf. https://jasp-stats.org/faq/what-does-jasp-stand-for/) is a user-friendly free open-source statistical analysis software. It includes a meta-analysis module. SPSS users will find many aspects of the interface familiar. - https://jasp-stats.org/ - Introductory Youtube video (in Portuguese): [Introdução ao JASP][3]   - **R and metafor package** - R is a free open-source software environment for statistical computing and graphics. It is a powerful command-line analysis software with a vast library of statistical packages. - https://www.r-project.org/ - Base R program with all functionalities but with only a very simple user interface. RStudio is recommended as a user interface instead. - https://www.rstudio.com/ - RStudio provides an integrated environment to use R with several helpful features. - There are numerous educational resources for learning about R and RStudio available online for free (e.g., https://education.rstudio.com/learn/beginner/) - http://www.metafor-project.org/ - metafor is one of the main meta-analysis packages for R (it is also what is used in JASP, under the hood). - https://github.com/ferreira-santos/metaforGUI#metaforgui - metaforGUI is a graphical user interface for the metafor package in R that allows you to run simple meta-analyses without coding and provides results outputs. In the meanwhile, JASP has included the meta-analysis module and you can do this using JASP as well.   - **Others** - Meta-Essentials: https://www.erim.eur.nl/research-support/meta-essentials/ - ProMeta: https://idostatistics.com/prometa3/ ---------- #### Installing software for the seminar For the practical part of the seminar you will need: - A spreadsheet editor (like Microsoft Excel, Google Sheets, etc.) - JASP - How to install JASP: https://jasp-stats.org/faq/how-do-i-install-jasp/ - Introductory Youtube video (in Portuguese): [Introdução ao JASP][4] (covers installation and software overview) - R and RStudio (optional) - Install R (base distribution): https://cran.rstudio.com/ - Install RStudio Desktop (free): https://rstudio.com/products/rstudio/download - Other things we will need (like installing required R packages) will be done during the seminar ---------- ### Additional resources: Below are a few references for topics that extend the seminar (not organized in any meaningful way; suggestions for additional references are welcome): - **Additional references** (beyond the Core references provided above): - Camilo, C., & Garrido, M. V. (2019). A revisão sistemática de literatura em psicologia: Desafios e orientações. *Análise Psicológica, 37*, 535–552. https://doi.org/10.14417/ap.1546 - Cohen, J. (1988). *Statistical power analysis for the Behavioral Sciences* (2nd ed.). Lawrence Erlebaum Associates. - Higgins, J. P. T., Thomas, J., Chandler, J., Cumpston, M., Li, T., Page, M. J., & Welch, V. A. (Eds.) (2019). *Cochrane Handbook for Systematic Reviews of Interventions version 6.0 (updated July 2019)*. Retgrieved from www.training.cochrane.org/handbook - Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. *Frontiers in Psychology, 4*. https://doi.org/10.3389/fpsyg.2013.00863 - Siddaway, A. P., Wood, A. M., & Hedges, L. V. (2019). How to do a systematic review: A best practice guide for conducting and reporting narrative reviews, meta-analyses, and meta-syntheses. *Annual Review of Psychology, 70*, 747–770. https://doi.org/10.1146/annurev-psych-010418-102803 - **Resources on internal/mini meta-analysis**: - Goh, J. X., Hall, J. A., & Rosenthal, R. (2016). Mini meta-analysis of your own sudies: Some arguments on why and a primer on how. *Social and Personality Psychology Compass, 10*, 535–549, https://doi.org/10.1111/spc3.12267 - Ueno, T., Fastrich, G. M., & Murayama, K. (2016). Meta-analysis to integrate effect sizes within an article: Possible misuse and Type I error inflation. *Journal of Experimental Psychology: General, 145*, 643-654. https://doi.org/10.1037/xge0000159 - **Scoping reviews**: - Colquhoun, H. L., Levac, D., O’Brien, K. K., Straus, S., Tricco, A. C., Perrier, L., Kastner, M., & Moher, D. (2014). Scoping reviews: Time for clarity in definition, methods, and reporting. *Journal of Clinical Epidemiology, 67*, 1291-1294. https://doi.org/10.1016/j.jclinepi.2014.03.013 - Munn, Z., Peters, M. D. J., Stern, C., Tufanaru, C., McArthur, A., & Aromataris, E. (2018). Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. *BMC Medical Research Methodology, 18*, 143. https://doi.org/10.1186/s12874-018-0611-x - Pham, M. T., Rajić, A., Greig, J. D., Sargeant, J. M., Papadopoulos, A., & McEwen, S. A. (2014). A scoping review of scoping reviews: Advancing the approach and enhancing the consistency. *Research Synthesis Methods, 5*, 371–385. https://doi.org/10.1002/jrsm.1123 - **Moderately advanced methods paper on meta-analysis of correlated designs**: - Ferreira-Santos, F. (2018). Meta-analysis of correlated designs (repeated measures) in the context of neural responses to facial expressions of emotion. *SAGE Research Methods Cases*. https://doi.org/10.4135/9781526434180 - **Advanced methods paper on synthesizing multiple meta-analyses**: - Tang, L. L., Caudy, M., & Taxman, F. (2013). A statistical method for synthesizing meta-analyses. *Computational and Mathematical Methods in Medicine, 2013*, 732989. https://doi.org/10.1155/2013/732989 - [Resources for Learning (and Teaching) How to Conduct Meta-Analyses][5] - http://mgto.org/metaanalysissyllabus (brilliant shared GDoc by Gilad Feldman) ---------- - When you get tired of meta-analysing, try some math rock: @[youtube](https://www.youtube.com/watch?v=cHVRjTe4_Gs) Covet on Bandcamp :) - https://covetband.bandcamp.com [1]: https://osf.io/5q7pa/wiki/Versions/ [2]: https://media.giphy.com/media/NFA61GS9qKZ68/giphy.gif [3]: https://youtu.be/pwOwqUUCsS8 [4]: https://youtu.be/pwOwqUUCsS8 [5]: https://docs.google.com/document/d/1oImg-ojUFqak5KyZ-ETD2qGvkvUgx8Ym6b8gG4GwfM8