Conference article

Classifying Multimodal Turn Management in Danish Dyadic First Encounters

Constanza Navarretta
University of Copenhagen, Copenhagen Denmark

Patrizia Paggio
University of Copenhagen, Copenhagen Denmark and University of Malta, Valletta Malta

Download article

Published in: Proceedings of the 19th Nordic Conference of Computational Linguistics (NODALIDA 2013); May 22-24; 2013; Oslo University; Norway. NEALT Proceedings Series 16

Linköping Electronic Conference Proceedings 154:15, p. 133-146

NEALT Proceedings Series 16:15, p. 133-146

Show more +

Published: 2013-05-17

ISBN: 978-91-7519-589-6

ISSN: 1650-3686 (print), 1650-3740 (online)

Abstract

This paper deals with multimodal turn management in an annotated Danish corpus of video recorded dyadic conversations between young people who meet for the first time. Conversation participants indicate whether they wish to give; take or keep the turn through speech as well as body behaviours. In this study we present an analysis of turn management body behaviours as well as classification experiments run on the annotated data in order to investigate how far it is possible to distinguish between the different types of turn management expressed by body behaviours using their shape and the co-occurring speech expressions. Our study comprises body behaviours which have not been previously investigated with respect to turn management; so that it not only confirms preceding studies on turn management in English but also provides new insight on how speech and body behaviours are used together in communication. The classification experiments indicate that the shape annotations of all kinds of body behaviour together with information about the gesturer’s co-occurring speech are useful to classify turn management types; and that the various behaviours contribute to the expression of turn features in different ways. Thus; knowledge of the different cues used by speakers in face-to-face communication to signal different types of turn shift provides the basis for modelling turn management; which is in turn key to implement natural conversation flow in multimodal dialogue systems.

Keywords

Multimodal Communication; Turn Management; Multimodal Corpora; Machine Learning

References

No references available

Citations in Crossref