Depression is a highly prevalent, disabling and costly disorder that is linked with considerably diminished role functioning and quality of life, medical comorbidity and mortality (1–4). Psychological therapies are one of the first-line treatments of depression, and in the past decades several hundreds of randomized trials have examined the effects of these therapies. Several types of therapy have been developed and found to be effective, including cognitive behavior therapy (5,6), behavioral activation therapy (7,8), interpersonal psychotherapy (9,10), problem-solving therapy (11,12), psychodynamic therapy (13,14), third-wave psychotherapies (15), and non-directive counseling (16). These therapies are not only offered in an individual format, but also in group, telephone-based, guided self-help, and increasingly without any professional support through the internet (17,18). They are offered to specific populations, such as women with post-partum depression, older adults, children and adolescents, patients with comorbid general medical disorders, student populations and inpatients (19). Many studies also compare psychological treatments with pharmacotherapy and combined therapy (20,21). Because of all these differences between therapies, target groups, and treatment formats, many different comparisons have been examined in randomized trials in the past decades. Psychotherapies have been compared with control conditions, such as waiting lists and care as usual. They have been compared with each other, with pharmacotherapy and combined treatment, and different treatment formats have been compared with each other. Psychotherapy for depression is undoubtedly the best examined field of psychotherapy for any mental health problem. Every year dozens of new trials are conducted and the number of trials has been increasing steadily over the years, with more trials being conducted every year compared to the previous year (22). At this moment more than 600 trials on psychotherapy for depression in adults are available and it can be expected that every two years another 100 trials will be published in the coming year. Because of this huge body of knowledge, it is important that the results of these studies are summarized and integrated in meta-analytic studies. Many meta-analyses have been conducted over the past years. However, with this large and increasing number of trials it is important to keep a good, up-to-date overview of this field. Such an overview makes it possible to follow new developments and conduct meta-analyses when sufficient studies are available for a specific therapy or comparison. It also saves considerable time and effort for researchers who want to conduct a specific meta-analysis. Furthermore, it guides future research as knowledge gaps are identified and the pooled outcomes can be used for estimating sample sizes of clinical trials. Moreover, policymakers can use this summarized knowledge to inform evidence-based decision making. In the past decades, we have built a database like this for trials on psychotherapies for depression (23), which has been updated since then every year. We have used this database over the past decade to publish several dozens of meta-analyses, examining most of the comparisons described above. Since we started with this database more than a decade ago, the methods for searching, selecting and rating of trials in this field have been improved considerably. We decided therefore to write a new methods paper in which we describe the methods for this meta-analytic database. Another reason to write this methods paper is that we conducted the searches in the bibliographical databases all over again with new, improved search strings (rated by two independent researchers). We have conducted and finished the searches up to January 1st, 2019 and will report the results of these searches. We have not yet finished the ratings of all included studies and the calculated effect sizes (except for selected groups of studies), but we will report the resulting studies and the comparisons that are examined in these studies. This will allow other researchers to select and rate these studies for possible meta-analyses. We will also report on how studies were rated and categorized. Methods Searches in bibliographical databases We searched in four bibliographical databases, PubMed, Embase, PsycINFO, and the Cochrane Register of Controlled Trials (CENTRAL). We developed extensive search strings to identify randomized trials examining the effects of psychotherapy for depression, compared to any other intervention or control condition. In these search strings we combined index terms and text words indicative of depression and psychotherapies, with filters for randomized controlled trials. The full search string for each database is presented in File 1. We also added the references of trials through other sources, such as our previous database, other meta-analyses, and contact with other researchers. All records from all sources were entered into Endnote, and duplicates were removed. All resulting records were checked by two independent researchers (PC and EK). If one of the two researchers indicated that a record possibly contained a study that met inclusion criteria, the full text of that paper was retrieved. The full texts of the papers were read by the same researchers. All searches were conducted up to January 1st, 2019. However, we will update these searches every year in order to keep the database up to date. In- and exclusion of studies We included in principle all randomized trials in which a psychotherapy condition was compared with any other condition. That could be another psychotherapy, another format or version of the same therapy, pharmacotherapy, a control condition (such as a waiting list, and care-as-usual), or any other comparison group, such as exercise. We also included studies comparing combined treatment of psychotherapy and pharmacotherapy with either of these alone. We defined psychotherapy according to Norcross (24): “Psychotherapy is the informed and intentional application of clinical methods and interpersonal stances derived from established psychological principles for the purpose of assisting people to modify their behaviors, cognitions, emotions, and/or other personal characteristics in directions that the participants deem desirable”. We allowed different treatment formats, including individual, group, telephone, guided self-help (through the internet or not), and couple therapy. We also included self-guided interventions without any professional support, but made a separate category of these (see below), because of the absence of human interaction. Therapies can be delivered by any therapist (psychologist, psychiatrist, nurse, social worker, etc., but also lay health counselors and paraprofessionals) as long as they were trained to deliver the therapy. Studies were excluded in the following cases: • The study did not explicitly state it was randomized. Open studies without a comparison condition were also excluded. • When depression was not an inclusion criterion. We allowed any definition of depression (e.g., a depressive disorder according to a diagnostic interview; scoring above a cut-off on a self-rating depression scale; subthreshold depression), but we excluded studies in general populations without any indication for depression, and studies in which both depression and anxiety was an inclusion criterion (so also patients with only anxiety were included). • Maintenance studies: Studies on treatments aiming at patients in treatment with the aim of preventing relapse or maintaining outcomes over time in addition to acute phase treatment were also excluded, because we considered this to be a different field of research (although this may be added in future updates of our database). • Studies in children and adolescents: We excluded these studies but we hope to be able to add these to the database in future updates of our database. Studies in which only a part of participants were below 18 years of age (for example between 16 and 25) were also excluded. • Dissertations were also excluded (we did include these in previous versions of our database), because these are typically small, underpowered trials, where the student also conducts the intervention. These studies have also not been peer-reviewed. • We excluded studies on interventions in which the specific effects of psychological treatment cannot be discerned. This includes for example trials on collaborative care in which patients are also receiving drugs and the intervention is aimed at optimizing drug treatment. Also studies on stepped care where the specific effects of psychotherapy cannot be distinguished from the rest of the intervention were excluded. • We excluded studies in which the psychological treatment was not aimed at depression. For example, trials in patients with addiction and depression who receive a treatment for addiction were excluded. The same is true for interventions aimed at improving adherence to drugs or for example at improving sleep problems in depressed patients with insomnia. • Studies in which insufficient data are reported to calculate effect sizes were excluded. As main outcome we used depressive symptomatology according to any depression measure (see paragraph on outcomes). If a study did not report the means, standard deviation and number of participants in each condition (which is needed to calculate the effect size directly), we checked whether the effect size could be calculated with other data reported in the paper. This includes effect sizes based on a dichotomous outcome (e.g., response, remission, clinically significant change), which can be transformed into an effect size, or for example a p-value or t-value that indicates the difference between the two conditions. If none of these data were reported, the study was excluded. Previously, we also included studies in which a general p-value was reported (e.g., the p for the difference between two groups was p<0.05), and in which the calculation of the effect size was based on this general p-value. Because this over-estimates the true effect size, we did not include these studies anymore in the current database. • We included studies in English, Spanish, German and Dutch. If studies were reported only in another language they were excluded. Categories of trials on psychotherapy for depression We distinguished the following categories of comparisons: 1. Psychotherapy versus pharmacotherapy; in this category all studies are included in which patients are randomized to either psychotherapy or pharmacotherapy (for more elaborate descriptions of psychotherapy and pharmacotherapy, see below). 2. Combined treatment of psychotherapy plus pharmacotherapy versus pharmacotherapy alone; these studies show the additional benefits of adding psychotherapy to pharmacotherapy alone. 3. Combined treatment versus psychotherapy alone; these studies show the additional benefits of adding pharmacotherapy to psychotherapy alone. 4. Combined treatment versus psychotherapy plus pill placebo; these studies show the exact contribution of the active medication to combined treatment. 5. Psychotherapy versus control conditions; these studies examine the effects of psychotherapies compared to control conditions such as waiting list, care-as-usual, pill placebo, and others. 6. Psychotherapy versus another psychotherapy; in these studies, the effects of one type of psychotherapy is compared to those of another type of psychotherapy (for definitions of the main types of psychotherapy, see below). 7. Comparisons of different treatment format; a considerable number of trials compares different treatment formats for one type of therapy, for example individual versus group therapy, individual versus guided self-help, group versus telephone, etc. All of these comparisons are clustered in this category of studies. 8. Psychotherapy in inpatients; there is a group of studies examining the effects of psychotherapy in inpatients; we did not want to combine this with other comparisons (e.g., psychotherapy versus control conditions), because both the patient populations and the comparison conditions (inpatient care) are too different from each other. Therefore, we made a separate category of trials for inpatients. We considered psychiatric hospitals as well as nursing homes as inpatient settings. General medical hospital settings were not considered to be inpatients (because the control conditions usually do not include as much psychological and psychosocial support as psychiatric wards and nursing homes offer). 9. Self-help interventions without any professional support; we made a separate category of these interventions, because they can be considered in a way to be psychological treatments, but do not have a human component. Furthermore, they are consistently showing smaller effects than psychological therapies with human contact (18,25). These self-help interventions are also indicated as unguided interventions. They are mostly internet-based, but can also work with self-help books or other media. 10. Cognitive bias modification (CBM); we also made a separate category of trials examining CBM, because this is a very different type of psychological treatment and the effects are either small or non-existent (26). 11. Other comparisons. All other comparisons in which a psychological therapy was compared with another condition (and that did not fit into any of the other categories and that met the other inclusion criteria), were placed in this category. It included for example trials in which two types of combined treatment (with different psychotherapies but the same medication) were compared with each other, studies in which two types of the same therapy but with a variation in content were examined, studies comparing therapy with exercise, etc. Data extraction In File 2 we have reported what characteristics are extracted from the included studies. We have given the characteristics of the participants (recruitment methods, type of diagnosis, target group), of the therapies (format and number of sessions of the therapies, for type of therapy see below), for studies including pharmacotherapy we rated the type and for studies including a control group we rated the type of control group. We also reported where the study was conducted and in which year it was published. We also developed a system to categorize types of therapy. We built on the categories of psychotherapies for depression that we developed in an earlier meta-analysis (23). In this study, we closely examined the therapies that were used in 91 comparative outcome studies on depression and categorized them into clusters of therapies of 5 or more studies. We formulated definitions of the major types of psychological treatment that were found, and checked with at least two independent researchers whether the interventions from the studies met these descriptions. That resulted in seven major types of psychotherapy. One of these (social skills training) was removed later because too few studies were available (for several we could not calculate effect sizes based on our recent system). This left six major types of psychotherapy: cognitive behavior therapy (CBT), behavioral activation therapy, problem-solving therapy, interpersonal psychotherapy, psychodynamic therapy, and non-directive therapy. We added one broad category of newer types of therapy, third-wave therapies, which has been tested in a considerable group of recent trials. We also added life review therapy, which has been examined as well in a growing number of trials. The definitions and descriptions of the categories of therapy are described in File 3. For several categories of therapy, we have also developed subcategories (for example for CBT we made subcategories for CBT according the Beck and colleagues (27), the “Coping with depression” course of Lewinsohn and colleagues (28), etc.). In order to keep the ratings as simple as possible we have not included this in the standard ratings for the database. Risk of bias assessment We assessed the validity of included studies using four criteria of the ‘Risk of bias’ assessment tool, developed by the Cochrane Collaboration (29). This tool assesses possible sources of bias in randomized trials, including the adequate generation of allocation sequence; the concealment of allocation to conditions; the prevention of knowledge of the allocated intervention (masking of assessors); and dealing with incomplete outcome data (this was assessed as positive when intention-to-treat analyses were conducted, meaning that all randomized patients were included in the analyses). Assessment of the validity of the included studies is conducted by two independent researchers, and disagreements were solved through discussion. We have not yet rated selective outcome reporting, because the number of trials reporting that the protocol for the study has been published before the start of the study was very limited. In future updates of the database we plan to include this item as well. While the Cochrane tool rates criteria with three possible scores (plus, minus, uncertain), we have rated studies as positive or negative (uncertain was rated as negative) because we were most interested in studies with low risk of bias. Studies with low risk of bias give the best estimate of the true effect size. In future updates we consider to follow the Cochrane tool and use three possible scores. An updated version of the Cochrane is currently being prepared. After this is definite, we will consider to use this new tool in the updates of our database. Outcome measures Depressive symptomatology is the main outcome in most studies. Any instrument measuring depression is allowed for inclusion in the meta-analyses that are based on the studies in our database. These symptoms can be used with a self-report instrument such as the Beck Depression Inventory/BDI (30), the BDI-II (31), or with a clinician-rated instrument such as the Hamilton Depression Rating Scale/ HAMD (32), or the Montgomery Åsberg Depression Rating Scale/ MADRS (33). We calculate effect sizes indicating the difference between two conditions at post-test (and if available at follow-up) as the difference between the means of the two conditions divided by the pooled standard deviation. Because a considerable number of studies have relatively small sample sizes we correct the effect size for small sample bias, and use Hedges’ g as effect size (41). If means and standard deviations are not reported, we calculate the effect size using dichotomous outcomes using the procedures from Borenstein and colleagues (42). If these are not available either, we use other statistics (such as t-value or p-value) to calculate the effect size. In order to calculate effect sizes, we use all measures examining depressive symptoms and pool them within each study, before pooling them across studies, so that each study only has one effect size. The effect sizes and their standard error are all calculated in the computer program Comprehensive Meta-Analysis (current version is 3.3070; CMA). In our previous database we have also rated other outcomes, such as quality of life (34), functional limitations (35), social support (36) and negative outcomes (37–39). Although these are important outcomes we have decided not to include these for all studies in our new database. Meta-analyses Although our database is mainly a resource for researchers who want to conduct meta-analyses, regardless of the exact methods used for the meta-analyses, we also describe the general methods for the meta-analyses we will conduct with the database. The overall methods have been described previously in a general manual that is available on this website (file 4) and can be downloaded and distributed for free (40). To calculate pooled mean effect sizes, we use the “meta” and “metafor” packages in R, but whenever needed we also use the program Comprehensive Meta-Analysis (current version is 3.3070; CMA). Because we have found considerable heterogeneity in previous meta-analyses, we use a random effects pooling model in all analyses. Numbers-needed-to-be-treated (NNT) are calculated using the formulae provided by Furukawa (43), in which the control group’s event rate is set at a conservative 19% (based on the pooled response rate of 50% reduction of symptoms across trials in psychotherapy for depression) (44). In meta-analyses in which no control group is used, we prefer the methods of Kraemer and Kupfer (45) to calculate the NNT, because the method of Furukawa requires an event rate for the control condition. As a test of homogeneity of effect sizes, we calculate the I2-statistic, which is an indicator of heterogeneity in percentages. A value of 0% indicates no observed heterogeneity, and larger values indicate increasing heterogeneity, with 25% as low, 50% as moderate, and 75% as high heterogeneity (46). We calculate 95% confidence intervals around I2 (47) in the “metafor” package in R, or when using CMA, we use the non-central chi-squared-based approach within the heterogi module for Stata (48). In addition, we calculate the prediction interval, which estimates where the true effects are to be expected for 95% of similar studies that might be conducted in the future (49). We test for publication bias by inspecting the funnel plot on primary outcome measures and by Duval and Tweedie’s trim and fill procedure (50), which yields an estimate of the effect size after the publication bias has been taken into account (as implemented in CMA). We also conduct Egger’s test of the intercept to quantify the bias captured by the funnel plot and to test whether it is significant. We always conduct sensitivity analyses in which we limit the analyses to studies with low risk of bias. Subgroup analyses, testing whether the effect sizes differ significantly from each in other in subgroups of studies, are conducted according to the mixed effects model (42), in which studies within subgroups are pooled with the random effects model, while tests for differences between subgroups are conducted with the fixed effects model. Metaregression analyses, examining whether the effect size is associated with characteristics of studies are conducted according to the Knapp-Hartung procedures using the restricted maximum likelihood approach as implemented in CMA (42). Other types of meta-analyses using the database Up to now we have described the conventional meta-analyses that we have conducted with our database. However, we are also involved in other types of meta-analyses, such as network meta-analyses, ‘individual patient data’ (IPD) meta-analyses, and IPD network meta-analyses. We have published our first network meta-analyses based on our database, and several more are currently in preparation. We have also conducted several IPD meta-analyses and are preparing several more. The IPD meta-analyses we have conducted focus on specific trials from our database, including trials comparing cognitive behavior therapy with antidepressants (51,52), combined treatment versus pharmacotherapy alone (53), internet-based guided self-help for depression (54), internet-based unguided self-help (17), and one on psychodynamic psychotherapy (55). Several more are in preparation. We have also conducted on IPD network meta-analysis (56). It goes beyond the scope of this paper to describe the methods for these innovative types of meta-analyses in detail. Selection and inclusion of studies The PRISMA flowchart describing the inclusion process of studies up to January 1, 2019, including reasons for exclusion, is presented in File 5. In our searches, we identified 21,976 records from PubMed (5,169), Embase (6,424), PsycINFO (3,461) and the Cochrane Register (6,922). After exclusion of duplicates 16,701 records were left. These were screened by two independent raters. A total of 13,275 records were excluded based on the title and abstract. We retrieved the full text of 2,553 papers. After reading these papers (again by two independent reviewers), 661 studies met the inclusion criteria and were included in the database. The references of these studies are given in File 6. The included studies A table with the 619 included studies is given in File 7. For each study we have indicated which of the 11 categories of comparisons that were presented earlier are examined. A total of 66 studies compared psychotherapy with pharmacotherapy, 47 compared combined treatment with pharmacotherapy alone, 28 compared combined treatment with psychotherapy alone, and 17 compared combined treatment with psychotherapy plus pill placebo. A further 310 studies compared psychotherapy with a control group, 103 compared two types of psychotherapy with each other, 31 examined inpatients, 44 studies were focused on unguided self-help interventions, 34 studies compared different treatment formats for the same intervention, 9 examined cognitive bias modification, and 93 examined another comparison that did not fall in one of the other categories. The majority of studies (N=486; 79%) included only one of the 11 categories of comparisons, 100 studies (16%) included two categories, 23 (4%) three categories, and 10 (2%) four categories. We are currently working on extracting the data as described in the Methods section. We collect the data from the studies as part of meta-analyses we conduct. That means that not all data from all studies are available all the time. At the moment we are working on collecting all data from all studies in the database, but that is work in progress and not yet finished. The largest set of studies, on psychotherapies versus control conditions, is up to date and is available in file 8. This file should be self-explanatory with the characteristics of the studies. The effect sizes and standard errors are calculated in CMA. This file contains the data of all 330 trials comparing a psychotherapy with a control group (411 comparisons), in which 33,430 patients are included (17794 in the psychotherapy groups and 15636 in the control groups). Output We started with the first version of this database in 2006, and since then we have published many meta-analyses examining many aspects of psychotherapies for depression. A full list of meta-analyses that have used this database, and that have been published in peer-reviewed scientific journals is given in File 9. Every few years we have also tried to give an overview of the published meta-analyses and their main findings. The most recent one was published in Canadian Psychology (File 10). The references of the papers on IPD meta-analyses that have been published are presented in a separate file (File 11). Discussion We developed a new version of a meta-analytic database of psychological treatments of depression, built on the systems and experience from our earlier database (23), but with completely new searches and improved methods. At this moment we have included 661 trials in the database covering several comparisons of psychotherapies with pharmacotherapy, with other therapies and in different formats. With this large number of trials, psychotherapy for depression is definitely the best studied type of psychotherapy for any mental health problem. The number of trials is also increasing quickly. Between 2011 and 2015 a total of 228 trials were conducted, which is more than we were able to include in our first version of our database across all years (although we did not include all categories of trials that we have currently included). And based on the trials published since 2016, this increase seems to continue in the coming years. Research on psychotherapy has over time also broadened from North America to Europe and is now spreading quickly over other parts of the world, including non-Western countries (57). With this fast-growing field, meta-analyses are becoming more and more important. We hope that this database will help researcher with the integration of the knowledge that emerges from these trials and to learn more about what works and for whom. References 1. Spijker J, De Graaf R, Bijl RV, Beekman ATF, Ormel J, Nolen WA. Functional disability and depression in the general population. Results from the Netherlands Mental Health Survey and Incidence Study (NEMESIS). Acta Psychiatr Scand. 2004;110(3):208–214. 2. Cuijpers P, Vogelzangs N, Twisk J, Kleiboer A, Li J, Penninx BW. Comprehensive meta-analysis of excess mortality in depression in the general community versus patients with specific illnesses. Am J Psychiatry. 2014 Apr;171(4):453–62. 3. Whiteford HA, Degenhardt L, Rehm J, Baxter AJ, Ferrari AJ, Erskine HE, et al. Global burden of disease attributable to mental and substance use disorders: findings from the Global Burden of Disease Study 2010. Lancet. 2013;382(9904):1575–86. 4. Steel Z, Marnane C, Iranpour C, Chey T, Jackson JW, Patel V, et al. The global prevalence of common mental disorders: a systematic review and meta-analysis 1980-2013. Int J Epidemiol. 2014;43(2):476–93. 5. Cuijpers P, Cristea IA, Karyotaki E, Reijnders M, Huibers MJ. How effective are cognitive behavior therapies for major depression and anxiety disorders? A meta-analytic update of the evidence. World Psychiatry. 2016;15(3):245–258. 6. Furukawa TA, Noma H, Caldwell DM, Honyashiki M, Shinohara K, Imai H, et al. Waiting list may be a nocebo condition in psychotherapy trials: A contribution from network meta-analysis. Acta Psychiatr Scand. 2014;130(3):181–92. 7. Ekers D, Webster L, Van Straten A, Cuijpers P, Richards D, Gilbody S. Behavioural activation for depression; an update of meta-analysis of effectiveness and sub group analysis. PLoS One. 2014;9(6):e100100. 8. Shinohara K, Honyashiki M, Imai H, Hunot V, Caldwell DM, Davies P, et al. Behavioural therapies versus other psychological therapies for depression. The Cochrane Library. 2013;16(10):CD008696. 9. Churchill R, Davies P, Caldwell D, Moore TH, Jones H, Lewis G, et al. Interpersonal, cognitive analytic and other integrative therapies versus treatment as usual for depression. The Cochrane Library. 2010;9:CD008703. 10. Cuijpers P, Donker T, Weissman MM, Ravitz P, Cristea IA. Interpersonal psychotherapy for mental health problems: A comprehensive meta-analysis. Am J Psychiatry. 2016;173(7):680–7. 11. Cuijpers P, de Wit L, Kleiboer A, Karyotaki E, Ebert DD. Problem-solving therapy for adult depression: An updated meta-analysis. Europ Psychiatry. 2018;48:27–37. 12. Malouff JM, Thorsteinsson EB, Schutte NS. The efficacy of problem solving therapy in reducing mental and physical health problems: A meta-analysis. Clin Psychol Rev. 2007;27(1):46–57. 13. Driessen E, Hegelmaier LM, Abbass AA, Barber JP, Dekker JJM, Van HL, et al. The efficacy of short-term psychodynamic psychotherapy for depression: A meta-analysis update. Clin Psychol Rev. 2015 Aug 1;42:1–15. 14. Leichsenring F, Rabung S. Effectiveness of long-term psychodynamic psychotherapy: A meta-analysis. JAMA. 2008;300(13):1551–1565. 15. Churchill R, Moore TH, Furukawa TA, Caldwell DM, Davies P, Jones H, et al. ’Third wave’cognitive and behavioural therapies versus treatment as usual for depression. The Cochrane Library. 2013;10:CD008705. 16. Cuijpers P, Driessen E, Hollon SD, van Oppen P, Barth J, Andersson G. The efficacy of non-directive supportive therapy for adult depression: A meta-analysis. Clin Psychol Rev. 2012 Jun;32(4):280–91. 17. Karyotaki E, Riper H, Twisk J, Hoogendoorn A, Kleiboer A, Mira A, et al. Efficacy of Self-guided Internet-Based Cognitive Behavioral Therapy in the Treatment of Depressive Symptoms: A Meta-analysis of Individual Participant Data. JAMA Psychiatry. 2017;74(4):351–359. 18. Richards D, Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev. 2012;32(4):329–342. 19. Cuijpers P, Karyotaki E, Reijnders M, Huibers MJ. Who benefits from psychotherapies for adult depression? A meta-analytic update of the evidence. Cognit Behav Ther. 2018;47(2):91–106. 20. Cuijpers P, Sijbrandij M, Koole SL, Andersson G, Beekman AT, Reynolds CF 3rd. Adding psychotherapy to antidepressant medication in depression and anxiety disorders: a meta-analysis. World Psychiatry. 2014 Feb;13(1):56–67. 21. Cuijpers P, Sijbrandij M, Koole SL, Andersson G, Beekman AT, Reynolds CF 3rd. The efficacy of psychotherapy and pharmacotherapy in treating depressive and anxiety disorders: A meta-analysis of direct comparisons. World Psychiatry. 2013;12(2):137–48. 22. Cuijpers P. Psychotherapies for adult depression: recent developments. Curr Opin Psychiatry. 2015;28(1):24–9. 23. Cuijpers P, van Straten A, Warmerdam L, Andersson G. Psychological treatment of depression: A meta-analytic database of randomized studies. BMC Psychiatry. 2008;8(1):36. 24. Campbell LF, Norcross JC, Vasquez MJ, Kaslow NJ. Recognition of psychotherapy effectiveness: The APA resolution. Psychotherapy. 2013;50(1):98. 25. Andersson G, Cuijpers P. Internet-based and other computerized psychological treatments for adult depression: A meta-analysis. Cogn Behav Ther. 2009;38(4):196–205. 26. Cristea IA, Kok RN, Cuijpers P. Efficacy of cognitive bias modification interventions in anxiety and depression: meta-analysis. Br J Psychiatry. 2015 Jan;206(1):7–16. 27. Beck AT, Rush AJ, Shaw BF, Emery G. Cognitive Therapy of Depression. 1st ed. New York: The Guilford Press; 1979. 425 p. 28. Lewinsohn PM, Antonuccio DO, Steinmetz J, Teri L. The coping with depression course: A psycho-educational intervention for unipolar depression, 1984. Eugene: Castalia Publishing Company; 1984. 29. Higgins JPT, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928. 30. Beck AT, Ward CH, Mendelson M, Mock J, Erbaugh J. An inventory for measuring depression. Arch Gen Psychiatry. 1961;4(6):561–71. 31. Beck AT, Steer RA, Brown GK. BDI-II. Beck Depression Inventory Second Edition. Manual. San Antonio: Psychological Corporation; 1996. 32. Hamilton M. A rating scale for depression. J Neurol Neurosurg Psychiatr. 1960;23(1):56–62. 33. Montgomery SA, Åsberg M. A new depression scale designed to be sensitive to change. Br J Psychiatry. 1979;134(4):382–389. 34. Kolovos S, Kleiboer A, Cuijpers P. Effect of psychotherapy for depression on quality of life: meta-analysis. Br J Psychiatry. 2016;209(6):460–8. 35. Renner F, Cuijpers P, Huibers MJH. The effect of psychotherapy for depression on improvements in social functioning: a meta-analysis. Psychol Med. 2014 Oct;44(14):2913–26. 36. Park M, Cuijpers P, van Straten A, Reynolds CF 3rd. The effects of psychotherapy for adult depression on social support: A meta-analysis. Cognit Ther Res. 2014 Dec 1;38(6):600–11. 37. Cuijpers P, Reijnders M, Karyotaki E, de Wit L, Ebert DD. Negative effects of psychotherapies for adult depression: a meta-analysis of deterioration rates. J Affect Disord. 2018; 38. Ebert DD, Donkin L, Andersson G, Andrews G, Berger T, Carlbring P, et al. Does Internet-based guided-self-help for depression cause harm? An individual participant data meta-analysis on deterioration rates and its moderators in randomized controlled trials. Psychol Med. 2016;46(13):2679. 39. Karyotaki E, Kemmeren L, Riper H, Twisk J, Hoogendoorn A, Kleiboer A, et al. Is self-guided internet-based cognitive behavioural therapy (iCBT) harmful? An individual participant data meta-analysis. Psychol Med. 2018;1–11. 40. Cuijpers P. Meta-Analyses in mental health research; A practical guide. Amsterdam: Pim Cuijpers Uitgeverij; 2016. 41. Hedges LV, Olkin I. Statistical methods for meta-analysis. San Diego, CA: Academic Press; 1985. 42. Borenstein, M., Hedges, L.V., Higgins, J.P.T., Rothstein, H.R. Introduction to Meta-Analysis. Chichester, UK: Wiley; 2009. 43. Furukawa TA. From effect size into number needed to treat. Lancet. 1999;353(9165):1680. 44. Cuijpers P, Turner EH, Koole SL, van Dijke A, Smit F. What is the threshold for a clinically relevant effect? The case of major depressive disorders. Depress Anxiety. 2014 May;31(5):374–8. 45. Kraemer HC, Kupfer DJ. Size of treatment effects and their importance to clinical research and practice. Biol Psychiatry. 2006 Jun 1;59(11):990–6. 46. Higgins JPT, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. 2003;327(7414):557–60. 47. Ioannidis JPA, Patsopoulos NA, Evangelou E. Uncertainty in heterogeneity estimates in meta-analyses. BMJ. 2007;335(7626):914–6. 48. Orsini N, Bottai M, Higgins J, Buchan I. Heterogi: Stata module to quantify heterogeneity in a meta-analysis. Statistical Software Components [Internet]. 2006; Available from: https://ideas.repec.org/c/boc/bocode/s449201.html 49. Borenstein M, Higgins J, Hedges LV, Rothstein HR. Basics of meta-analysis: I2 is not an absolute measure of heterogeneity. Res Synth Meth. 2017;8(1):5–18. 50. Duval S, Tweedie R. Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics. 2000;56(2):455–63. 51. Weitz ES, Hollon SD, Twisk J, van Straten A, Huibers MJH, David D, et al. Baseline depression severity as moderator of depression outcomes between cognitive behavioral therapy vs pharmacotherapy: An individual patient data meta-analysis. JAMA Psychiatry. 2015 Sep 23;72(11):1102–9. 52. Vittengl JR, Jarrett RB, Weitz E, Hollon SD, Twisk J, Cristea I, et al. Divergent Outcomes in Cognitive-Behavioral Therapy and Pharmacotherapy for Adult Depression. Am J Psychiatry. 2016;173(5):481–90. 53. Weitz E, Kleiboer A, van Straten A, Hollon SD, Cuijpers P. Individual patient data meta-analysis of combined treatments versus psychotherapy (with or without pill placebo), pharmacotherapy or pill placebo for adult depression: a protocol. BMJ open. 2017;7(2):e013478. 54. Karyotaki E, Ebert DD, Donkin L, Riper H, Twisk J, Burger S, et al. Does guided internet-based interventions result in clinically relevant changes for patients with depression? An individual participant data meta-analysis. Clin Psychol Rev. 2018;63:80–92. 55. Driessen E, Abbass AA, Barber JP, Gibbons MBC, Dekker JJ, Fokkema M, et al. Which patients benefit specifically from short-term psychodynamic psychotherapy (STPP) for depression? Study protocol of a systematic review and meta-analysis of individual participant data. BMJ open. 2018;8(2):e018900. 56. Furukawa TA, Efthimiou O, Weitz ES, Cipriani A, Keller MB, Kocsis JH, et al. Cognitive-Behavioral Analysis System of Psychotherapy, Drug, or Their Combination for Persistent Depressive Disorder: Personalizing the Treatment Choice Using Individual Participant Data Network Metaregression. Psychotherapy Psychosom. 2018;87(3):140–153. 57. Cuijpers P, Karyotaki E, Reijnders M, Purgato M, Barbui C. Psychotherapies for depression in low-and middle-income countries: a meta-analysis. World Psychiatry, 17, 90–101.