Complex mapping of words and gestures in TED Talks

The article explores the interdependence of the verbal and the physically embodied – here gesture-related – subsystems in the creation of meaning in a sample of TED Talks (www.ted.com), an increasingly popular genre for scientific popularization largely exploited in education. My goal is to identify and illustrate possible indices of complexity in the mapping of words with gestures, thus paving the way for a better understanding of the role of different semiotic resources in the talks and, ultimately, contributing to the development of multimodal literacy. In fact, the hybrid nature of the genre legitimises a holistic approach to the analysis of its discourse as a complex multisemiotic system. Multimodal ensembles (Kress 2003; 2009; 2010) are viewed as a special case of complex systems, and modal density (Norris 2009) and modal coherence (cf. Valeiras Jurado 2017) as indices of complexity therein. Data description is based on multimodal transcription through an integrated method (Lazaraton 2004), which makes it possible to advance hypotheses about the interpretation of different gestures (NcNeill 1992). Indeed, several gestures in the talks under analysis complement verbal information in no redundant ways and appear to serve various functions on different discourse levels, both locally and more globally, and in more or less predictable (hence more complex, context-dependent) ways.