> **Abstract** Diverse groups are often said to be less susceptible to decision errors resulting from herding and polarization. Thus, the fact that many modern interactions happen in a digital world, where homophily brings people together into filter bubbles, is a disturbing yet puzzling result. But online interactions are also characterized by unprecedented scale, where thousands of individuals can exchange ideas simultaneously. Evidence in social learning and collective intelligence however suggests that small groups tend to do better in complex information environments. Here, we adopt the well-established framework of social learning theory (from the fields of ecology and cultural evolution) to explore the causal link between diversity and performance as a function of group size. In this pre-registered study, we experimentally manipulate both group diversity and group size, and measure individual and group performance in a realistic geo-political forecasting task. We find that diversity hinders the performance of individuals in small groups, but improves it in large groups. Furthermore, aggregating opinions of modular crowds composed of small independent but homogeneous groups achieves better results than using non-modular diverse ones. The results are explained by greater conflict of opinion in diverse groups, which negatively impacts small (but not large) groups. The present work sheds light on the causal mechanisms underlying the success (or lack thereof) of diverse groups. The results also suggest that diversity research can benefit from applying theory and methods used in collective information processes. > **Pre-registration** The study was pre-registered at asPredicted.org. You can see the pre-registered information at http://aspredicted.org/blind.php?x=8di37y