Are torso movements during speech timed with intonational phrases?
Stefanie Shattuck-Hufnagel, P. Lin Ren, E. Tauscher, MIT
It is well known that speakers often move their arms, hands, heads and parts of their faces in conjunction with their speech. Recent studies indicate that these movements are often temporally aligned with the accented syllables in the spoken utterances they accompany, forming a kind of gestural analogue to the accentual aspect of phrase-level prosody. But less attention has been given to the question of how body movements might relate to the other major aspect of phraselevel prosody: word grouping or phrasing. In this study we examined the temporal alignment of torso movement with the intonational phrases identifiable in short speech samples from two speakers selected from a corpus of academic lecture videos. Results show that a) the two speakers differ substantially in their torso movements during speech, b) the speaker who showed the most frequent use of left-right movement of the shoulders timed these movements to coincide to a notable extent with his intonational phrases, and c) torso movements are sometimes timed in other ways, such as during silences between spoken phrases, possibly in conjunction with a change in topic. Overall, these observations support the hypothesis that both of the grammatically-significant aspects of phrase-level prosody, i.e. prominence and phrasing, can have analogues in the organization of body movements that accompany speech production. This strengthens the view that speech production planning involves the coordination not only of gestures within the vocal tract with each other and with prosodic structure, but also of gestures and movements of other parts of the body, whose contribution to the communicative act merits further exploration.