Main content

Home

Menu

Loading wiki pages...

View
Wiki Version:
**Method** The present meta-analysis aims to be reproducible and fully transparent. In order to accomplish this goal, we make the analysis script and the data for this study publicly available in the [Data analytic plan][1] and [Data][2] sections. All materials of this meta-analysis can be found via the [Material][3] section (including search strategy, coding sheet, p-curve disclosure table, analysis workflow). Our meta-analysis is pre-registered on PROSPERO [link] under protocol number [NUMBER]. The availability of our materials will allow other researchers to carry out further analyses and check the robustness of our conclusions. We believe that this approach will increase the credibility of our findings. We also intend to build a cumulative scientific knowledge base by constructing a database in which the data on different strategies are mapped, allowing other researchers to contribute. To accomplish this aim, we will build a public repository that will allow researchers and experts on stress to add their data (Tsuji, Bergmann, & Cristia, 2014) to construct a community-augmented meta-analysis. Any changes to the pre-registration will be fully disclosed on the Appendix A in [Materials][4] using the same template provided by Moreau and Gamble (2020). We intend to map the methodological quality and possible biases in this field, carry out several sensitivity analyses with respect to our main results, and control for prognostic methodological factors related to the magnitude and precision of the reported effects in subgroup analyses. **Inclusion criteria, and search strategy** To frame our research question in a structured way, we followed the Participants, Intervention, Comparator, and Outcome (PICO) Framework (Schardt et al., 2007). We chose to only include studies on participants that are adults (people aged 18 years or older). We only included two interventions (self-administered mindfulness and biofeedback based on Heart-Rate Variability). For designs comparing groups, we included effects based on a comparison to an active control condition, meaning that participants are subjected to an alternative treatment for stress reduction (e.g., progressive muscle relaxation), or to a passive control condition (participants are in an untreated comparison group; e.g., waitlist control). If there were more than one comparator in the same study (i.e., presence of both an active and a passive control group), we chose the contrast with the active control group. The main outcome of this meta-analysis was the level of stress after intervention, divided into three components (affect, cognition, and physiology). We relied on self-reported measures (for the affective and the cognitive component and for the longer-term affective consequences of stress) taken at post-test (immediate and/or delayed) of the two groups (experimental and control). For the physiological component we relied on changes in the activation of the hypothalamic-pituitary adrenal axis (e.g., via cortisol levels, heart-rate) taken at post-test (immediate and/or delayed) of the two groups (experimental and control). After framing our search question, we conducted a preliminary search to pilot the planned search strategy and coding scheme (on a dozen eligible articles randomly chosen from each strategy category). To guarantee the reproducibility of the literature search, we followed the recommendations by Maggio et al. (2011), who provided guidelines to specify 1) which databases we used 2) which search terms we used, 3) which Medical Subject Headings (MeSH) or thesaurus terms we used, 4) on which dates we searched, and 5) what our search limits were. We searched the literature using the following sources: ProQuest, (an online platform which included research coming from APA PsycArticles, APA Psycinfo, ProQuest Dissertations & Theses Global‎), PubMED, and Scopus. The first author (AS) conducted the literature search and excluded articles which were not relevant to the aim of the present meta-analysis. Screening by title and abstract was carried out using Rayyan QCRI (Ouzzani, Hammady, Fedorowicz, & Elmagarmid, 2016), a web and mobile app for systematic reviews and meta-analyses. He then manually searched reference lists for more citations and unpublished reports and screened reference lists in published articles as well. Finally, we asked for unpublished results via social networks (Facebook Groups and Twitter), via relevant mailing lists (SPSP, EASP, ESCAN), and via listservs of an association of clinical and health psychology (EACLIPT). Furthermore, to have a wider coverage of the literature, we contacted authors that have published studies on the topic to see if they have any unpublished research, in-progress manuscripts, or in-press manuscripts (see our templates in Appendices B and C in [Materials][5]). We also included articles from previous meta-analyses that we may have missed with other search methods. The inclusion criteria for our meta-analysis were as follows: 1) We included published articles, preprint articles, working papers, dissertations, and books (we included only studies published in English), 2) we included both randomized control trials and observational studies that estimated the effect of (or exposure to) one of the two stress regulation strategies, 3) we included studies that measured at least one of the three components of the stress response, which we included in the meta-analysis (physiological, affective, and cognitive), or at least that measured the affective consequences of stress, and 4) the participants of the study had to be humans. Studies were excluded if 1) the paper was a review or a meta-analysis, 2) participants were adolescents (under 18 years of age), 3) a common-metric effect size reflecting a bivariate effect or relationship could not be computed, or 4) the two types of intervention included in our meta-analysis (self-administered mindfulness & biofeedback based on Heart-Rate Variability) were combined with other active treatment (e.g., cognitive behavioral therapy, physical exercise). We then added sub-exclusion criteria related to the two stress-regulation strategies. For self-administered mindfulness, we excluded studies 1) which involved the presence of an instructor, 2) which cannot be self-administered, and 3) which involve meditation practices in groups with other people. For each study on self-administered mindfulness, we coded the source from which participants got the instructions for doing the mindfulness exercises (e.g., an audio source or via a website). For biofeedback, we excluded from the meta-analysis any kind of biofeedback that was not based on Heart-Rate Variability. A PRISMA flow chart of the overall literature search and inclusion procedure is shown in Appendices D and E in [Materials][6]. **Coding and Data Preparation** The coding was carried out by two independent coders, with the first author coding all the included literature and the second coder coding at least 20% of the data for cross-checking. The coding process was cross-checked after the first 10% and after the second 10%. To evaluate the inter-rater reliability, Cohen’s Kappa and inter-rater agreement were calculated. We double-coded a subset of the data, including the part of the statistics and characteristics of the individual studies. Following the guidelines of Landis and Koch (1977), we considered an agreement of κ > 0.60 for metric or multinomial variables acceptable. Consensus between two coders was reached through discussion in case of disagreement. We extracted data for the following variables: publication year, the number of citations of the paper, country, journal name, reported overall N, gender ratio, publication status, reported effect sizes, total N, cell means, SDs and Ns, test statistic, degrees of freedom, the type of effect (e.g., main omnibus, main bivariate, interaction), the reported p-value, the design, the type of population, the nationality of participants, whether individual differences were taken into account or not, the name of stress test used in experiments, the duration of the intervention, the type of intervention, the category of stress-regulation strategy (self-administered mindfulness, biofeedback), the type of control group (active, passive), the type of population, whether it was one of the components of stress (affective, cognitive, or physiological) or an affective consequences of stress, the number of measurements, and the instrument employed to assess stress levels. We converted all the relevant bivariate effect sizes (ES) to a common metric (Hedges’ g), which is the standardized mean difference corrected for small sample bias (Hedges & Olkin, 1985). We also extracted the reported degrees of freedom as a more accurate estimate of the sample size because dropping of participants may have not been properly disclosed. For designs with two or more experimental groups, we checked the accuracy of the sample size in two ways: First, we looked at the reported Ns of each group, then we computed the total number of participants. If the sum matched the overall sample size, the respective group cell size was used. If that was not the case, the overall N was used, assuming a balanced design. In the case that the authors provided information only for the total sample size, we assumed equal cell sizes. For within-participants designs, a conservative correlation between the measurements of r = .50 was assumed. Lastly, we conducted a sensitivity analysis varying the assumed correlation from .10 to .90, in steps of .20 in order to determine the impact on the overall ES estimate. **[POST PRE-REGISTRATION EDIT]** After a first pre-registration, we started our search and found too many irrelevant articles, due to the fact that the search was carried out in full text. In this way we realized that we were retrieving articles that mentioned one of our keywords, but that weren't necessarily focused on those keywords. Thus, to make the search more precise, we searched for the terms present in the search strategy in the title or in the abstract of the articles. To do this, for each database we selected in "advanced search" the option that allowed to search for the keywords of the search strategy only in the abstract and in the title of the articles. [1]: https://osf.io/58waj/ [2]: https://osf.io/dpj4r/ [3]: https://osf.io/dpj4r/ [4]: https://osf.io/dpj4r/ [5]: http://%5BMaterials%5D [6]: https://osf.io/dpj4r/
OSF does not support the use of Internet Explorer. For optimal performance, please switch to another browser.
Accept
This website relies on cookies to help provide a better user experience. By clicking Accept or continuing to use the site, you agree. For more information, see our Privacy Policy and information on cookie use.
Accept
×

Start managing your projects on the OSF today.

Free and easy to use, the Open Science Framework supports the entire research lifecycle: planning, execution, reporting, archiving, and discovery.