Loading wiki pages...

Wiki Version:
There will be a live interaction moment at Friday the 17th of April at 16:00 o'clock Berlin time: https://us04web.zoom.us/j/73674802984 If you are planning to join, it would be great if you drop me a message in advance so I have a sense what the chances are of people attending: wimpouw@gmail.com. Abstract: Communicative hand gesticulations are tightly coupled to quasi-rhythmic aspects of speech (i.e., prosody), such that emphatically stressed movements co-occur with stressed speech. Psychologists have characterized this multimodal synchrony as a preplanning process governed by a sophisticated cognitive timing mechanism acquired only later in development. Recently it has however been found that acoustic markers of emphatic stress arise naturally during steady-state phonation when upper-limb movements impart physical impetus on the body, most likely affecting acoustics via lower vocal tract activity. Such findings ground a rudimentary form of gesture-speech synchrony in biomechanics. The proposed mechanism for gesture-induced force-transmissions onto speech, is that muscles around the lower vocal tract (i.e, the trunk) are implicated in upper limb movement, leading to sudden increases in alveolar (lung) pressure. Firstly, this is because respiratory-related muscles are implicated in anticipatory postural adjustments which counteract upper limb perturbances onto the bodies center of pressure. Secondly, it is known that forces produced locally at the upper limb, can affect more distal muscle activity in the body due to resonances in the pre-stressed myofascial-skeletal system. Thus, gestures can form functional synergies with the respiratory system in attaining vocalization targets. However, so far biomechanical gesture research has been performed on steadystate vocalizations, from which we cannot infer whether gesture-induced physical impetus affects more complex vocalizations. Also, a direct investigation is needed into whether lower vocal tract dynamics is indeed the link that connects gesture and speech acoustics. In this confirmatory study (N = 29) participants uttered consonant-vowel CV (/pa/) mono-syllables in rhythmic fashion while moving the upper limbs (or not). We show that chest-wall kinematics is affected by (especially high- impetus) gesturing when vocalizations occur near peaks in physical impetus. We further show that gesture-induced moment of impetus increases the amplitude envelope of speech, while not similarly affecting the Fundamental Frequency (F0). Finally, we find tight relations between chestkinematic activity and vocalization, even in the absence of movement, but more strong relations are found when upper limb movement is present. The current findings provide a more compelling case that speech acoustics is modulated by functional biomechanical linkages between hand gesture and the lower vocal tract. *The uploaded file contains a slide and video presentation of the talk merged in one mp4 video. There will be a moment for online exchange of comments and questions. I will update on that soon. ** There is a misalignment in the audio at alter sections of the talk, I am sorry for this. Please also note that there is a preprint of this study online already: https://psyarxiv.com/rnpav For questions: w.pouw@psych.ru.nl
OSF does not support the use of Internet Explorer. For optimal performance, please switch to another browser.
This website relies on cookies to help provide a better user experience. By clicking Accept or continuing to use the site, you agree. For more information, see our Privacy Policy and information on cookie use.

Start managing your projects on the OSF today.

Free and easy to use, the Open Science Framework supports the entire research lifecycle: planning, execution, reporting, archiving, and discovery.