Main content
Many Labs 2: Investigating Variation in Replicability Across Sample and Setting /
Many Labs 2: Investigating Variation in Replicability Across Sample and Setting
- Richard A. Klein
- Michelangelo Vianello
- Fred Hasselman
- Byron Gregory Adams
- Reginald B. Adams, Jr.
- Sinan Alper
- Mark Aveyard
- Jordan Axt
- Mayowa Babaloia
- Štěpán Bahník
- Mihaly Berkics
- Michael Jason Bernstein
- Daniel R. Berry
- Olga Bialobrzeska
- Konrad Bocian
- Mark Brandt
- Robert Busching
- Huajian Cai
- Fanny Cambier
- Katarzyna Cantarero
- Cheryl L Carmichael
- Zeynep Cemalcilar
- Jesse J. Chandler
- Jen-Ho Chang
- Armand Chatard
- Eva CHEN
- Winnee Cheong
- David C Cicero
- sharon coen
- Jennifer A. Coleman
- Brian Collisson
- Morgan Conway
- Katherine S. Corker
- Paul G Curran
- Fiery Cushman
- Ilker Dalgar
- William E. Davis
- Maaike de Bruijn
- Marieke de Vries
- Thierry Devos
- Canay Doğulu
- Nerisa Dozo
- Kristin Dukes
- Yarrow Dunham
- Kevin Durrheim
- Matthew Easterbrook
- Charles R. Ebersole
- John Edlund
- Alexander Scott English
- Anja Eller
- Carolyn Finck
- Miguel-Ángel Freyre
- Mike Friedman
- Natalia Frankowska
- Elisa Maria Galliani
- Tanuka Ghoshal
- Steffen Robert Giessner
- Tripat Gill
- Timo Gnambs
- Angel Gomez
- Roberto Gonzalez
- Jesse Graham
- Jon Grahe
- Ivan Grahek
- Eva Green
- Kakul Hai
- Matthew Haigh
- Elizabeth L. Haines
- Michael P. Hall
- Marie E. Heffernan
- Joshua A. Hicks
- Petr Houdek
- Marije van der Hulst
- Jeffrey R. Huntsinger
- Ho Phi Huynh
- Hans IJzerman
- Yoel Inbar
- Åse Innes-Ker
- William Jimenez-Leal
- Melissa-Sue John
- Jennifer Joy-Gaba
- Roza Kamiloglu
- Andreas Kappes
- Heather Kappes
- Serdar Karabati
- Haruna Karick
- Victor N. Keller
- Anna Kende
- Nicolas Kervyn
- Goran Knezevic
- Carrie Kovacs
- Lacy Elise Krueger
- German Kurapov
- Jaime Kurtz
- Daniel Lakens
- Lili Lazarevic
- Carmel Levitan
- Neil Lewis, Jr.
- Samuel Lins
- Esther Maassen
- Angela Maitner
- Winfrida Malingumu
- Robyn Mallett
- Satia Marotta
- Jason McIntyre
- Janko Međedović
- Taciano L. Milfont
- Wendy Morris
- Andriy Myachykov
- Sean Murphy
- Koen Ilja Neijenhuijs
- Anthony J. Nelson
- Felix Neto
- Austin Lee Nichols
- Susan L. O'Donnell
- Masanori Oikawa
- Gabor Orosz
- Malgorzata Osowiecka
- Grant Packard
- Rolando Pérez
- Boban Petrovic
- Ronaldo Pilati
- Brad Pinter
- Lysandra Podesta
- Monique Pollmann
- Anna Dalla Rosa
- Abraham M. Rutchick
- Patricio Saavedra M.
- Airi Sacco
- Alexander K Saeri
- Erika Salomon
- Kathleen Schmidt
- Felix Schönbrodt
- Maciek Sekerdej
- David Ricardo Sirlopu
- Jeanine Skorinko
- Michael A. Smith
- Vanessa Smith-Castro
- Agata Sobkow
- Walter J. Sowden
- Philipp Spachtholz
- Troy G. Steiner
- Jeroen Stouten
- Chris N. H. Street
- Oskar Sundfelt
- Ewa Szumowska
- Andrew Tang
- Norbert K. Tanzer
- Morgan Tear
- Jordan Theriault
- Manuela Thomae
- David Torres
- Jakub Traczyk
- Joshua M. Tybur
- Adrienn Ujhelyi
- Marcel A.L.M. van Assen
- Anna van 't Veer
- Alejandro Vásquez-Echeverría
- Leigh Ann Vaughn
- Alexandra Vázquez
- Diego Vega
- Catherine Verniers
- Mark Verschoor
- Ingrid Voermans
- Marek A. Vranka
- Cheryl Welch
- Aaron Wichman
- Lisa A. Williams
- Julie A. Woodzicka
- Marta Katarzyna Wronska
- Liane Young
- John M. Zelenski
- Brian A. Nosek
Date created: | Last Updated:
: DOI | ARK
Creating DOI. Please wait...
Category: Uncategorized
Description: We conducted preregistered replications of 28 classic and contemporary published findings with protocols that were peer reviewed in advance to examine variation in effect magnitudes across sample and setting. Each protocol was administered to approximately half of 125 samples and 15,305 total participants from 36 countries and territories. Using conventional statistical significance (p < .05), fifteen (54%) of the replications provided evidence in the same direction and statistically significant as the original finding. With a strict significance criterion (p < .0001), fourteen (50%) provide such evidence reflecting the extremely high powered design. Seven (25%) of the replications had effect sizes larger than the original finding and 21 (75%) had effect sizes smaller than the original finding. The median comparable Cohen’s d effect sizes for original findings was 0.60 and for replications was 0.15. Sixteen replications (57%) had small effect sizes (< .20) and 9 (32%) were in the opposite direction from the original finding. Across settings, 11 (39%) showed significant heterogeneity using the Q statistic and most of those were among the findings eliciting the largest overall effect sizes; only one effect that was near zero in the aggregate showed significant heterogeneity. Only one effect showed a Tau > 0.20 indicating moderate heterogeneity. Nine others had a Tau near or slightly above 0.10 indicating slight heterogeneity. In moderation tests, very little heterogeneity was attributable to task order, administration in lab versus online, and exploratory WEIRD versus less WEIRD culture comparisons. Cumulatively, variability in observed effect sizes was more attributable to the effect being studied than the sample or setting in which it was studied.
Add important information, links, or images here to describe your project.