After conducting a replication a key step is to integrate your new results with the original findings. This is accomplished through **meta-analysis**. By combining results with meta-analysis you will be able to provide a more precise estimate of the effect size of interest. You will also be able to estimate the heterogeneity of effect size estimates (how different your new data is from the original data) and explore for possible moderators. Meta-analysis can be complex, but a number of tools are available that can help make this critical step relatively straightforward. **This page is still in draft form--need to extend directions on using OpenMeta[Analyst] to include meta-regression and subgroup analysis.** ---------- [Open Meta\[Analyst\]][1] ---------- Open Meta[Analyst] is a free tool that can make meta-analysis very straightforward. It runs in Windows and is actually a front-end for the powerful R statistical programming environment. You can download Open Meta[Analyst] [here][2]. Open Meta-analysis uses the non-central t distribution for calculating the integrated effect size--that may seem like goblity-gook, but what it means is that is uses one of the best methods for combining results together. **Some university firewalls flag this program as a virus! It is not a virus, but that can make it difficult to download. Unfortunately, the developers of the software do not seem to have a solution for this problem.** Using Open Meta[Analyst] is relatively straightforward. These steps walk you through conducting a meta-analysis focusing on a standardized effect size estimate for the different between two groups (Cohen's d, which, when corrected for bias is often called Hedge's g). 1. Click “Create new project” 2. Choose “SMD”. You’ll be asked to confirm that you want to use the standardized mean difference and also asked to name the dependent variable. 3. In the spreadsheet that appears, enter the data for the original study. Specifically, enter the sample size, mean, and standard deviation for each group of the original study. 4. Once the required data is entered, the columns labelled SMD, lower, and upper will be calculated and filled in. The SMD column is Cohen's d corrected for bias (also called Hedge's g). The lower and upper columns report the lower and upper bounds of the 95% CI. In the figure below, the result is: dunbiased,= -0.926, 95% CI[-1.438, -0.414]. 5. Now enter your own data (or additional studies for synthesis). Enter one line per study, and for each study give a name and enter sample size, mean, and standard deviation for each group. Continue entering data, one line per study, until you have assembled all the studies for synthesis. 6. Click the button that shows a forest plot in Black and White. This opens a dialog box for choosing meta-analysis options. 7. In the dialog box that appears, click "OK" (generally the default options are just fine). 8. An output box will appear with text output and a forest plot. Marked at number 8 is the overall effect size estimate and CI along with a test for heterogeneity. 9. Marked at 9 is the corresponding forest plot. Right click to access options to save it as a .png or .pdf file 10. Here is the forst plot. Note that it also includes the overall effect size estimate and CI along with a test for heterogeneity of effect size. Clearly, Meta-Analysis is complex and you'll want to read up on how to select studies, what the options are for analysis, and what the output means. With a good understanding of the basics of meta-analysis, Open Meta[Analyst] can be extremely useful. ![Step 1 for Open Meta Analyst][3] ![Step 2 for Open Meta Analyst][4] ---------- Other Meta-Analysis Tools ---------- Here is a list of other tools you can use for Meta-Analysis: * [**ESCI**][5] by Geof Cumming is an set of Excel Spreadsheets for conducting meta-analysis for two-group comparisons. Sheets are available for raw data input or for standardized group differences (Cohen's d). Subgroup analysis can be conducted. ESCI also uses the non-central t distribution for calculating overall CIs, which is one of the better approaches. ESCI is very easy to use, but is limited in the types of meta-analyses it can conduct and in the sample sizes per study (no more than ~100/group) * **[The Meta-Analysis Calculator][6]** is a fully online tool for meta-analysis. Can handle lots of different effect size measures. Doesn't seem to be actively maintained, though, and seems to be a bit buggy. * **[Meta-Essentials][7]** is a free Excel-based set of tools for conducting Meta-Analysis using a wide range of effect size measures. It is powerful and has a good user guide. This tool uses a weighted variance approach to calculating the overall CI--this will give you longer CIs than you'll get with ESCI or OpenMeta[Analyst], but it is a reasonable approach, and this keeps the worksheet simple and easy to use (no macros). In my experience, Meta-Essentials is the most likely to work right out of the box, with none of the sample-size limitations of ESCI or the download and platform issues of OpenMeta[Analyst]. It also has the best user guide. So if you are struggling with your first meta-analysis, this is probably the best tool. * [Practicel Meta-Analysis Effect Size Calculator][8] - this set of online tools does not conduct a meta-analysis, but it can help you calculate effect sizes and CIs for a wide range of different types of study designs. [1]: http://www.cebm.brown.edu/openmeta/ [2]: http://www.cebm.brown.edu/openmeta/ [3]: https://mfr.osf.io/export?url=https://osf.io/x24kd/?action=download&direct&mode=render&initialWidth=684&childId=mfrIframe&format=800x800.jpeg [4]: https://mfr.osf.io/export?url=https://osf.io/vksyw/?action=download%26direct%26mode=render&initialWidth=684&childId=mfrIframe&format=800x800.jpeg [5]: http://www.latrobe.edu.au/psychology/research/research-areas/cognitive-and-developmental-psychology/esci [6]: http://www.lyonsmorris.com/ma1/ [7]: http://www.erim.eur.nl/research-support/meta-essentials/ [8]: http://www.campbellcollaboration.org/escalc/html/EffectSizeCalculator-Home.php