A crucial aspect of understanding others’ thoughts and feelings is to infer meaning from rich sensory signals, such as human vocalizations (e.g., laughter, cries, moans, screams). One commonplace assumption is that the more intense the affective state underlying the expression, the higher the ability to make sense of it. The evidence for this conjecture is sparse, and despite coherent theoretical predictions, attempts to map the relation between affective intensity and inferred meaning are controversial. Using a new database of non-speech expressions varying from low to peak emotion intensity, we test the influence of intensity on evaluation. In three experiments, participants (N = 90) classified and rated emotion and intensity, affective valence and arousal, and perceived authenticity. We reveal a paradoxical role of intensity: Although emotional intensity and arousal can be inferred unambiguously across the range of expressed intensities, hedonic value and emotion category reveal a ‘perceptual sweet spot’ at moderate to strong emotions. Perception is most ambiguous at peak emotion. The VIVAE stimulus set used in this study is openly available from http://doi.org/10.5281/zenodo.4066235