Main content

Home

Menu

Loading wiki pages...

View
Wiki Version:
### **Title: A methodological review to develop a list of bias items used to assess reviews incorporating network meta-analysis: Protocol and rationale** Lunny C, Cochrane Hypertension Review Group and the Therapeutics Initiative, University of British Columbia, Canada Tricco AC, Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Unity Health Toronto, 209 Victoria Street, East Building, Toronto, ON, M5B 1T8, Canada; Dalla Lana School of Public Health & Institute of Health Policy, Management, and Evaluation, University of Toronto; Queen's Collaboration for Health Care Quality Joanna Briggs Institute Centre of Excellence, Queen’s University Veroniki A, School of Education, University of Ioannina, Ioannina, Greece Dias, S, Centre for Reviews and Dissemination, University of York, York, UK Hutton, B, Ottawa Hospital Research Institute, Ottawa, Canada. Ottawa University, School of Epidemiology and Public Health, Ottawa, Canada Salanti G, Institute of Social and Preventive Medicine, University of Bern, Switzerland Wright J, Cochrane Hypertension Review Group and the Therapeutics Initiative, University of British Columbia, Canada White IR, MRC Clinical Trials Unit at UCL, London, UK Whiting P, Population Health Sciences, Bristol Medical School, University of Bristol * Corresponding author: Carole Lunny, MPH, PhD, Cochrane Hypertension, Therapeutics Initiative, Department of Anesthesiology, Pharmacology & Therapeutics, Faculty of Medicine, University of British Columbia, 2176 Health Science Mall, Vancouver, BC, Canada, V6T 1Z3, carole.lunny@ti.ubc.ca #### **1.0 INTRODUCTION** ****1.1 Reviews with network meta-analysis (NMA)**** Reviews with network meta-analysis (NMAs) have gained popularity due to their ability to provide comparative effectiveness of multiple treatments for the same condition [1]. Reviews with NMA have grown in number. Between 1997 and 2015, 771 NMAs were published in 336 journals from 3459 authors and 1258 institutions in 49 countries [2]. More than three-quarters (n = 625; 81%) of these NMAs were published in the last 5-years. Many organisations such as the National Institute for Health and Care Excellence (NICE) in the UK, the World Health Organization (WHO), and Canadian Agency for Drugs and Technologies in Health (CADTH) conduct NMAs as they represent the best available evidence to inform clinical practice guidelines [3-5]. We adopt a broad definition of NMAs, specifically: a review that aims to, or intends to, simultaneously synthesise more than two heath care interventions of interest. Reviews that intend to compare multiple treatments with an NMA but then find that the assumptions are violated (e.g. a disconnected network, or studies are too heterogeneous to combine) and that NMA is not feasible, will also be included in our definition. **1.2 Systematic reviews and bias** Evidence shows that biased results from poorly designed and reported studies can mislead decision-making in healthcare at all levels [6-9]. If a review is at risk of bias and inappropriate methods are used, the validity of the findings can be compromised [10-12]. Evaluating how well a review has been conducted is essential to determining whether the findings are relevant to patient care and outcomes. Several empirical studies have shown that bias can obscure the real effects of a treatment [13-16]. Being able to appraise reviews with NMA is central to informed decision-making in patient care. The systematic procedures required to conduct a systematic review (e.g. double and independent data extraction and comparison of the extractions) help mitigate the risk of bias. However, bias can also be introduced when interpreting the reviews findings. For example, review conclusions may not be supported by the evidence presented, the relevance of the included studies may not have been considered by review authors, and reviewers may inappropriately emphasise results on the basis of their statistical significance [17]. A well-conducted systematic review draws conclusions that are appropriate to the included evidence and can therefore be free of bias even when the primary studies included in the review have high risk of bias. **1.3 Tools for critical appraisal** Tools are available for most study designs to make risk of bias assessment easier for a knowledge user (e.g., healthcare practitioners, policymakers, patients [18]). Many tools and checklists can be used either when conducting a systematic review (quality of conduct), when assessing how well a study has been described (reporting), or when knowledge users want to assess the risk of bias in the conclusions of a review. The methodological quality of studies (i.e., how well the study is conducted) is often confused with reporting quality (i.e., how well authors report their methodology and results). A risk of bias assessment is an assessment of review limitations, which focus on the potential of those methods to bias the study findings [17]. **1.4 Critical appraisal of reviews with pairwise meta-analysis** More than 40 tools have been identified [19, 20] for critically appraising the quality of reviews with pairwise meta-analysis. AMSTAR AMSTAR (A MeaSurement Tool to Assess the methodological quality of systematic Reviews) [21] and the OQAQ (Overview Quality Assessment Questionnaire [22]) have been identified as the most commonly used, and they follow a simple checklist format [20, 23]. AMSTAR has been recently updated to AMSTAR 2, which aims to evaluate how reviews are planned and conducted [24]. The ROBIS (Risk Of Bias In Systematic reviews) tool is designed to assess the risk of bias in systematic reviews with or without pairwise meta-analysis [23]. The ROBIS tool involves assessment of methodological features in reviews known to increase the risk of bias in review conclusions. Domain-based assessment tools require a careful reading and thoughtful analysis of the study to adequately rate risk of bias, instead of simply identifying keywords reported in the article, as usually made in a checklist type of assessment. **1.5 Critical appraisal of reviews with NMA** For critically appraising reviews with NMA, several checklists exist. To assess reporting quality, the PRISMA statement extension for reviews incorporating network meta-analysis (PRISMA-NMA) [25] or the National Institute for Health and Care Excellence Decision Support Unit checklist (NICE-DSU) [26] can be used. To assess quality of conduct, the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) checklist [27] can be used. However, many quality assessment tools are not created rigorously. To be rigorous, they must follow a series of systematic steps [28, 29]. As a quality of conduct tool, the ISPOR checklist [27] did not follow the methodology proposed by Whiting [29] for creating a systematically developed quality tool. Due to important methodological advances in the field of NMA, the ISPOR, published in 2014, is also outdated. As Table 1 shows, several tools are designed with different purposes; some for assessing reporting quality, and some for assessing quality of conduct but none are designed to assess risk of bias in NMAs. **Table 1. Tools and checklists to aid in systematic review conduct, or to assess the reporting or methodological quality of a review** @[OSF](3q8mn) **1.6 Importance of assessing reviews with NMA for biases** Given that the aim of a systematic review is to reduce bias introduced due to methods used in combining RCTs, a specific tool aiming to identify bias in NMAs is needed. A systematic review is a synthesis of the literature. Bias is reduced in systematic reviews through the systematic identification, appraisal, synthesis and, if relevant, statistical aggregation, of all studies on a specific topic according to predetermined and explicit methodology. For a practicing clinician, researcher or policymaker, choosing from conflicting NMAs is impossible without a risk of bias assessment tool. An empirical evaluation identified 28 NMAs on treatment for rheumatoid arthritis with considerable discrepancies observed across data extracted and risk of bias assessments of included RCTs, assessment of heterogeneity, and eligibility of outcomes. Concerns with each of these issues, as well as discrepancies between NMAs, leaves uncertainty regarding which of the biologics has the greatest treatment effect. Evidence shows that biased results from poorly designed and reported studies can mislead decision-making in healthcare at all levels. A methods study, which can be extrapolated to NMAs, found that only ~3% of pairwise meta-analyses are both methodologically sound and clinically useful. A risk of bias assessment is an assessment of review limitations, which focus on the potential of review methods to bias research findings. If a review is at risk of bias and inappropriate methods are used, the validity of the findings can be compromised. Evaluating how well a study has been conducted is essential to determining whether the findings are valid and reliable for use to guide patient care and outcomes. Several studies have shown that bias can obscure the real effects of a treatment. Being able to appraise reviews with NMA using a targeted tool for this purpose is central to evidenced-based decision-making in patient care. The novel elements in reviews with NMA require a bias assessment tool specifically tailored to NMAs. A review of published NMAs showed that 75% were low methodological quality based on a tool to assess pairwise meta-analysis (i.e. AMSTAR 2 [24]). This assessment was unable to assess biases that are specific to the conduct of NMA – such as biases introduced when NMA assumptions are not met, and when effect modifiers are present. Building a risk of bias tool helps to disseminate the newest developments and best practices in the conduct of NMAs. **1.7 Systematic process for tool development** A comprehensive and systematic process should be used to develop a rigorous risk of bias tool for assessing NMAs, as outlined in Whiting et al.’s “Framework for Developing Quality Assessment Tools” [29]. The first step is to: (1) conduct a systematic search of biases that can inform the assessment of the validity and reliability of NMAs and prepare a pilot list of items, (2) create a draft tool, (3) obtain expert opinion on the draft tool through Delphi exercises, and (4) pilot test and refine the tool [29]. No review has comprehensively and systematically listed and categorised all items related to quality of bias in NMAs. Such a list will inform a new tool to assess the risk of bias in NMAs, and potentially other reporting or quality tools which are being updated. #### **2.0 OBJECTIVE** Our objective is to conduct a methodological review to develop a list of items relating to bias in NMAs with the goal of developing a risk of bias tool to assess NMAs. #### **3.0 METHODS AND ANALYSIS** We will follow the methodology proposed by Whiting [29], Sanderson [37] and Page [7] for creating systematically developed lists of quality items. **3.1 Eligibility criteria** There will be two types of studies included. Study type 1 are articles that present and describe items related to bias, reporting, or methodological quality of reviews with NMA. Items related to reporting will be retained because they can potentially be translated into a risk of bias item. For example, in the PRISMA-P guideline [38], one item asks whether study PICO [Population, Interventions, Comparisons, Outcomes] characteristics were used as criteria for determining study eligibility. Reporting of all outcomes in a protocol may prevent authors from only selecting outcomes that are statistically significant when publishing their systematic review. This PRISMA-P reporting item can then be translated into a bias item related to the “selective reporting” of outcomes [39]. Study type 2 are studies that assess the methodological quality in a sample of reviews with NMA. Study Type 1 will meet any of these inclusion criterion: • Articles describing items related to bias or methodological quality in reviews with NMA (e.g. Dias 2018 [40]); tools that only assess general aspects of systematic reviews without focusing specifically on NMA will be excluded (e.g. AMSTAR [21], AMSTAR 2 [24] or ROBIS [17]). • Articles describing editorial standards for reviews with NMA (e.g. similar to the Cochrane MeCIR (Methodological standards for the conduct of new Cochrane Intervention Reviews) standards for systematic reviews [30]). • Articles describing items related to reporting quality in reviews with NMA (e.g. PRISMA-NMA [25]). • Articles identifying or addressing sources of bias and variation in NMA and published after PRISMA-NMA in 2014. Study Type 2 will meet any of these inclusion criterion: • Articles assessing the methodological quality (or risk of bias) of reviews with NMA (i.e. a sample of NMAs are assessed for methodological quality; e.g. Chambers 2015 [41]) using criteria that focus specifically on aspects of NMA not just on general aspects of systematic reviews. We will include articles with any publication status and in any language, and where the co-authors are not fluent in the language, Google Translate will be used. If through our main search, we identify a systematic review encompassing the eligible articles, or one aspect of the eligible article, we will use the results of the systematic review and only include primary studies published subsequent to the systematic review. For example, a review by Laws et al. in 2019 [5] identified all guidance documents for conducting an NMA from countries throughout the world. We therefore would not search for guidance documents published before the last search date of this review. **3.2 Search strategy** We will search Ovid MEDLINE (January 1946 to June 2020), the Cochrane library as well as the following grey literature databases: the EQUATOR Network (http://www.equator-network.org/reportingguidelines/), Dissertation Abstracts, websites of evidence synthesis organisations (Campbell Collaboration Cochrane Multiple Treatments Methods Group, CADTH, NICE-DSU, Health Technology Assessment International (HTAi), Pharmaceutical Benefits Advisory Committee, Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen, European Network for Health Technology Assessment, Guidelines International Network, ISPOR, International Network of Agencies for Health Technology Assessment, and JBI) as well as methods collections (i.e. Cochrane Methodology Register, AHRQ Effective Health Care Program). We will validate the MEDLINE strategy by using the PubMed IDs of ten included studies (identified by experts prior to our eligibility screening) and evaluating whether the strategy identified the PMIDs (Appendix 1). A systematic search strategy will be developed by two methodologists (CL, PW) without limitations to publication type, status, language, or date to identify existing tools or articles. An information specialist will check the search strategy for MEDLINE Ovid and assess it using the PRESS (Peer Review Electronic Search Strategies) guidance [42]. The full search strategies for all databases and websites can be found in Appendix 1. To identify other potentially relevant studies, we will examine the reference lists of included studies. We will ask experts in methods for NMA to identify articles missed by our search. We will contact authors of abstracts to retrieve the full report or poster. We will search the reference section of a bibliometric study of reviews with NMAs [43] and extract the name of the journals that publish NMAs. We will then contact their editors in chief and ask if they have any in-house editorial standards for reviews with NMA. **3.3 Process for screening, data extraction and analysis** The eligibility criteria will be piloted in Microsoft Excel by two reviewers independently on a sample of 25 citations retrieved from the search to ensure consistent application. After high agreement (>70%) is achieved, the Covidence [44] web-based tool (https://www.covidence.org) will be used by two reviewers to independently screen the citations based on the eligibility criteria. Disagreements will be discussed until consensus is reached. A third reviewer (CL) will arbitrate if disagreements cannot be resolved. The data extraction form will be piloted by reviewers independently on a sample of five included papers to ensure consistent coding. Two independent authors will extract data on the characteristics of the studies, and items. Any disagreements will be arbitrated by a third author. **3.4 Data extraction** The sources will first be categorised by the type of article coded as per our inclusion criteria. A table of tool characteristics will be developed with the following headings: first author, year; type of tool (tool, scale, checklist, or domain-based tool); whether the tool is designed specific topic areas (specify); number of items; domains within the tool; whether the item relates to reporting or methodological quality (or other concepts such as precision, acceptability); how items and domains within the tool are rated; methods used to develop the tool (e.g. review of items, Delphi study, expert consensus meeting); and the availability of an “explanation and elaboration” [7]. Data will be extracted on items that are potentially relevant to the risk of bias or quality of reviews with NMAs. Items will be initially extracted verbatim. **3.5 Data analysis** The following steps will be used when analysing items: 1. Map to ROBIS domains Items will be mapped to ROBIS domains (study eligibility criteria; identification and selection of studies; data collection and study appraisal; and synthesis and findings) and specific items within the domains. The rationale for mapping items to ROBIS is that it is the only tool to assess risk of bias in reviews. Items that do not clearly map to the existing ROBIS domains will be listed separately and grouped by similar concept. New domains may be created if items do not fit well into the established ROBIS domains. 2. Split items so that each item only covers a single concept Two or more concepts grouped in one item will be split so that each item covers a single concept. A rationale as to why the item was split will be described. For example, PRISMA-NMA item 15 (“Specify any assessment of risk of bias that may affect the cumulative evidence (e.g., publication bias, selective reporting within studies)”) will be split into two items because this item is represented by two items in ROBIS in the synthesis and findings domain, namely “4.5 Were the findings robust, e.g. as demonstrated through funnel plot or sensitivity analyses?” and “4.6 Were biases in primary studies minimal or addressed in the synthesis?”. 3. Group similar items Items that are conceptually similar will be grouped together and noted with the source. We will classify items as relating to bias or other aspect of quality (e.g. reporting). When relevant, items related to reporting will be reworded into items related to bias in NMA review conclusions. 4. Omit duplicate items (but keep these in a column in the table for transparency) If items are worded vaguely or are unexplained, we will use an iterative process to interpret the item and ensure there is a mutual understanding of the item between authors when coding. The process will be iterative, and if any gaps in items related to bias in reviews of NMA are identified, a new item will be inferred. The final list of items deemed unique will be retained. We will reword items as signalling questions, where an answer of “yes” suggests absence of bias. We will provide examples to illustrate the items and write a rationale and description of each item. We will count the number of sources and unique items included. We will summarise the characteristics of included tools in tables and figures. We will calculate the median and interquartile range (IQR) of items across all tools and tabulate the frequency of different biases identified in the tools. 3.6 Patient and Public Involvement Patients or the public were not involved in the design of our research protocol. #### **4.0 ETHICS AND DISSEMINATION** No ethics approval was required as no human subjects were involved. Our research aims to develop a list of items related to bias in the goal of developing the first tool for assessing risk of bias in the findings of reviews with NMA. We plan to publish the full study in a peer reviewed journal, and disseminate the findings via social media (Twitter, Facebook, and author affiliated websites). Patients, healthcare providers and policy makers need the highest quality evidence to make decisions about which treatments should be used in healthcare practice. Being able to critically appraise the findings of reviews with NMA is central to evidenced-based decision-making in patient care. #### **References** 1. Ioannidis, J.P., Integration of evidence from multiple meta-analyses: a primer on umbrella reviews, treatment networks and multiple treatments meta-analyses. Cmaj, 2009. 181(8): p. 488-493. 2. Li, L., et al., The global research collaboration of network meta-analysis: a social network analysis. PloS One, 2016. 11(9). 3. Kanters, S., et al., Use of network meta-analysis in clinical guidelines. Bull World Health Organ 2016. 94(10): p. 782-784. 4. Eiring, Ø., Brurberg, K.G., Nytrøen, K., Nylenna, M., Rapid methods including network meta-analysis to produce evidence in clinical decision support: a decision analysis. Systematic reviews, 2018. 7(1): p. 168. 5. Laws, A., et al., A Comparison of National Guidelines for Network Meta-Analysis. Value Health, 2019. 22(10): p. 1178-1186. 6. Egger, M., Smith, G.D., Schneider, M., Minder, C., Bias in meta-analysis detected by a simple, graphical test. BMJ, 1997. 315(7109): p. 629-634. 7. Page, M.J., J.E. McKenzie, and J.P.T. Higgins, Tools for assessing risk of reporting biases in studies and syntheses of studies: a systematic review. BMJ Open, 2018. 8(3): p. e019703. 8. Page, M.J., et al., Bias due to selective inclusion and reporting of outcomes and analyses in systematic reviews of randomised trials of healthcare interventions. Cochrane Database Syst Rev, 2014(10). 9. Petticrew, M., Why certain systematic reviews reach uncertain conclusions. BMJ, 2003. 326(7392): p. 756-758. 10. Greco, T., et al., The attractiveness of network meta-analysis: a comprehensive systematic and narrative review. Heart Lung Vessel, 2015. 7(2): p. 133. 11. Jansen, J.P. and H. Naci, Is network meta-analysis as valid as standard pairwise meta-analysis? It all depends on the distribution of effect modifiers. BMC medicine, 2013. 11(1): p. 159. 12. Li, T., et al., Network meta-analysis-highly attractive but more methodological research is needed. BMC Med, 2011. 9(1): p. 79. 13. Dechartres, A., Altman, D.G., Trinquart, L., Boutron, I., Ravaud, P., Association between analytic strategy and estimates of treatment outcomes in meta-analyses. JAMA, 2014. 312(6): p. 623-630. 14. Hart, B., A. Lundh, and L. Bero, Effect of reporting bias on meta-analyses of drug trials: reanalysis of meta-analyses. Bmj, 2012. 344: p. d7202. 15. Klassen, T.P., M.L. Lawson, and D. Moher, Language of publication restrictions in systematic reviews gave different results depending on whether the intervention was conventional or complementary. J Clin Epi, 2005. 58(8): p. 769-776. e2. 16. Mhaskar, R., et al., Critical appraisal skills are essential to informed decision-making. Indian J Sex Transm Dis AIDS, 2009. 30(2): p. 112. 17. Whiting, P., et al., ROBIS: Tool to assess risk of bias in systematic reviews-Guidance on how to use ROBIS. Available at)(Accessed March 26, 2018) http://www. bristol. ac. uk/media-library/sites/social-community-medicine/robis/robisguidancedocument. pdf View in Article, 2016. 18. Canadian Institutes of Health Research. Knowledge User Engagement. 2016. 19. Lunny, C., et al., Toward a comprehensive evidence map of overview of systematic review methods: paper 2-risk of bias assessment; synthesis, presentation and summary of the findings; and assessment of the certainty of the evidence. Syst Rev, 2018. 7(1): p. 159. 20. Whiting, P., et al., Chapter 4. Phase 2: Review of existing quality assessment tools for systematic reviews. , in Evidence to inform the development of ROBIS, a new tool to assess the risk of bias in systematic reviews. 2013: Available from http://www.robis-tool.info [accessed 11/06/2020]. p. p. 22–35. 21. Shea, B.S., et al., Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol 2007. 7(1): p. 10. 22. Oxman, A.D. and G.H. Guyatt, Validation of an index of the quality of review articles. J Clin Epidemiol, 1991. 44(11): p. 1271-8. 23. Pieper, D., et al., Overviews of reviews often have limited rigor: a systematic review. J Clin Epidemiol, 2012. 65(12): p. 1267-73. 24. Shea, B.J., et al., AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ, 2017. 358: p. j4008. 25. Hutton, B., Salanti, G., Caldwell, D.M., Chaimani, A., Schmid, C.H., Cameron, C., Ioannidis, J.P., Straus, S., Thorlund, K., Jansen, J.P., Mulrow, C., Catalá-López, F., Gøtzsche, P.C, Dickersin, K., Boutron, I., Altman, D.G., Moher, D. , The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations. Ann Intern Med, 2015. 162(11): p. 777-84. 26. Dias, S., et al., NICE DSU Technical Support Document 4: Inconsistency in Networks of Evidence Based on Randomised Controlled Trials. 2011, National Institute for Health and Care Excellence (NICE) Decision Support Unit (DSU): London, UK. 27. Jansen, J.P., et al., Indirect treatment comparison/network meta-analysis study questionnaire to assess relevance and credibility to inform health care decision making: an ISPOR-AMCP-NPC Good Practice Task Force report. Value Health, 2014. 17(2): p. 157-173. 28. Moher, D., et al., Guidance for developers of health research reporting guidelines. PLoS Med, 2010. 7(2). 29. Whiting, P., et al., A proposed framework for developing quality assessment tools. Syst Rev, 2017. 6(1): p. 204. 30. Chandler, J., et al., Methodological standards for the conduct of new Cochrane Intervention Reviews. Sl: Cochrane Collaboration, 2013. 31. Guyatt, G., et al., GRADE guidelines: 1. Introduction—GRADE evidence profiles and summary of findings tables. Journal of clinical epidemiology, 2011. 64(4): p. 383-394. 32. Puhan, M.A., et al., A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis. Bmj, 2014. 349: p. g5630. 33. Brignardello-Petersen, R., et al., Advances in the GRADE approach to rate the certainty in estimates from a network meta-analysis. Journal of clinical epidemiology, 2018. 93: p. 36-44. 34. Nikolakopoulou, A., et al., CINeMA: An approach for assessing confidence in the results of a network meta-analysis. PLoS Medicine / Public Library of Science, 2020. 17(4): p. e1003082. 35. Phillippo, D.M., et al., Threshold analysis as an alternative to GRADE for assessing confidence in guideline recommendations based on network meta-analyses. Annals of internal medicine, 2019. 170(8): p. 538-546. 36. Moher, D., et al., Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS med, 2009. 6(7): p. e1000097. 37. Sanderson, S., I.D. Tatt, and J. Higgins, Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography. International journal of epidemiology, 2007. 36(3): p. 666-676. 38. Moher, D., et al., Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev, 2015. 4(1): p. 1. 39. Higgins, J.P.T., et al., Cochrane Handbook for Systematic Reviews of Interventions version 6.1. 2020: Cochrane. 40. Dias, S., et al., Chapter 8 Validity of network meta-analyses. In: Network meta-analysis for decision-making. 2018: John Wiley & Sons. 41. Chambers, J.D., et al., An assessment of the methodological quality of published network meta-analyses: a systematic review. PloS one, 2015. 10(4). 42. McGowan, J., et al., PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement. Journal of Clinical Epidemiology, 2016. 75: p. 40-46. 43. Ban, J.K., et al., History and publication trends in the diffusion and early uptake of indirect comparison meta-analytic methods to study drugs: animated coauthorship networks over time. BMJ open, 2018. 8(6): p. e019110. 44. Innovation, V.H., Covidence systematic review software. 2016, Veritas Health Innovation Melbourne, VIC.
OSF does not support the use of Internet Explorer. For optimal performance, please switch to another browser.
Accept
This website relies on cookies to help provide a better user experience. By clicking Accept or continuing to use the site, you agree. For more information, see our Privacy Policy and information on cookie use.
Accept
×

Start managing your projects on the OSF today.

Free and easy to use, the Open Science Framework supports the entire research lifecycle: planning, execution, reporting, archiving, and discovery.