Date created: | Last Updated:

Category: Project

Description: Many commercial products use algorithms to recognize affect based on speech prosody (i.e., voice acoustics). These algorithms are typically trained on enacted or labeled speech samples collected in lab settings. However, they are used to infer affective experiences occurring in everyday life. Here, we investigate whether the experience of affective states can be predicted from speech samples collected using smartphones in naturalistic settings. In two field studies (experimental Study 1: N = 409; observational Study 2: N = 687), we collected 25,403 speech samples from participants along with their self-reported affective experiences. Machine learning analyses show that prosody reveals limited affective information (rmd = .17) and is outperformed by semantic content (rmd = .33). Our findings demonstrate the importance of semantic content and challenge whether previously reported prediction performances for affective expression from prosody in controlled settings generalize to the recognition of subjective affective experience in naturalistic settings.