Published: 2013-05-17
ISBN: 978-91-7519-589-6
ISSN: 1650-3686 (print), 1650-3740 (online)
This paper deals with multimodal turn management in an annotated Danish corpus of video recorded dyadic conversations between young people who meet for the first time.
Conversation participants indicate whether they wish to give; take or keep the turn through speech as well as body behaviours. In this study we present an analysis of turn management body behaviours as well as classification experiments run on the annotated data in order to investigate how far it is possible to distinguish between the different types of turn management expressed by body behaviours using their shape and the co-occurring speech expressions. Our study comprises body behaviours which have not been previously investigated with respect to turn management; so that it not only confirms preceding studies on turn management in English but also provides new insight on how speech and body behaviours are used together in communication. The classification experiments indicate that the shape annotations of all kinds of body behaviour together with information about the gesturer’s co-occurring speech are useful to classify turn management types; and that the various behaviours contribute to the expression of turn features in different ways. Thus; knowledge of the different cues used by speakers in face-to-face communication to signal different types of turn shift provides the basis for modelling turn management; which is in turn key to implement natural conversation flow in multimodal dialogue systems.
Allwood; J.; Cerrato; L.; Jokinen; K.; Navarretta; C. & Paggio; P. (2007). The MUMIN coding scheme for the annotation of feedback; turn management and sequencing. Multimodal Corpora for Modelling
Human Multimodal Behaviour. Special Issue of the International Journal of Language Resources and Evaluation; 41(3–4); 273–287.
Argyle; M. &Cook; M. (1976). Gaze and mutual gaze. Cambridge University Press; Cambridge; UK.
Boersma; P. & Weenink; D. (2009). Praat: doing pho- netics by computer. Retrieved May 1; 2009; from http://www.praat.org/.
Campbell. N. (2009). An audio-visual approach to mea- suring discourse synchrony in m ultimodal conversa- tion data. In Proceedings of Interspeech 2009; pp. 12–14.
Cohen. J. (1960). A coefficient of agreement for nom- inal scales. Educational and Psychological Mea- surement; 20(1):37–46.
Cowley; S. J. (1998). Of Timing; Turn-Taking; and Conversations. Journal of Psycholinguistic Research; 27(5); 541–571.
Du-Babcock; B. (2003). A comparative analysis of individual communication processes in small group behavior between homogeneous and heterogeneous groups. In Proceedings of the 68th Association of Business Communication Convention; pages 1–16; Albuquerque; New Mexico; USA.
Duncan; S. Jr. and D.W. Fiske; D. W. (1977). Face-to-face in- teraction. Erlbaum; Hillsdale; NJ.
Duncan; S. Jr. (1972). Some Signals and Rules for Taking Speaking Turns in Conversations. Journal of Personality and Social Psychology; 23(2); 283–292.
Esposito; A.; Campbell; N.; Vogel; C.; Hussain; A. & Nijholt; A. (eds). (2010). Development of Multi- modal Interfaces: Active Listening and Synchrony; volume 5967 of LNCS. Springer Verlag. Ford; C. E. & Thompson; S. A. (1996). Interactional Units in Conversation: Syntactic; Intonational; and Pragmatic Resources for the Management of Turns. In E. Ochs; E.A. Schegloff; and S.A. Thompson; editors; Interaction and Grammar; pp. 134–184. Cambridge University Press; Cambridge. Hadar; U. Steiner; T. J. & Clifford Rose; F. (1984). The timing of shifts of head postures during conversa- tion. Human Movement Science; 3(3); 237–245.
Jokinen; K. (2011). Turn taking; utterance density; and gaze patterns as cues to conversational activ- ity. In Proceedings of ICMI-MMI; Alicante; Spain; November.
Kendon; A. (1967). Some functions of gaze-direction in social interaction. Acta Psychologica; 26; 22–63.
Kipp; M. (2004). Gesture Generation by Imitation - From Human Behavior to Computer Character An- imation. Ph.D. thesis; Saarland University; Saar- bruecken; Germany; Boca Raton; Florida; disserta- tion.com.
Tanaka; H. (2008). Communication strategies and cul- tural assumptions: An analysis of French-Japanese business meetings. In S. Tietze; editor; International Management and Language; pp 154–170. Rout- ledge; New York; NY.
Yngve; V. (1970). On getting a word in edgewise. In Papers from the sixth regional meeting of the Chicago Linguistic Society; pp. 567–578.
Lu; J.; Allwood; J.& Ahlse´n; E. Under publication. A study on cultural variations of smile based on em- pirical recordings of Chinese and Swedish first en- counters. In D. Heylen; M. Kipp; and P. Paggio; editors; Proceedings of the workshop on Multimodal Corpora at ICMI-MLMI 2011; Alicante; Spain; Nov.
Maynard; S. (1987). Interactional functions of a nonver- bal sign: Head movement in Japanese dyadic casual conversation. Journal of Pragmatics; 11:589–606.
Navarretta; C. & Paggio; P. (2012). Verbal and non-verbal feedback in different types of interactions. In Proceedings of LREC 2012; pp. 2338–2342; Is- tanbul Turkey; May.
Navarretta; C.; Ahlse´n; E.; Allwood; J.; Jokinen; K. &; Paggio; P. (2011). Creating Comparable Multimodal Corpora for Nordic Languages. In Proceedings of the 18th Conference Nordic Conference of Compu- tational Linguistics; pages 153–160; Riga; Latvia; May 11-13.
Navarretta; C.; Ahlse´n; E.; Allwood; J.; Jokinen; K. & Paggio; P. (2012) Feedback in Nordic first- encounters: a comparative study. In Proceedings of LREC 2012; pp. 2494–2499; Istanbul Turkey; May.
O’Connell; D. C.; Kowal; S. &Kaltenbacher; E. (1990). Turn-Taking: A Critical Analysis of the Research Tradition. Journal of Psycholinguistic Research; 19(6):345–373.
Paggio; P. & Navarretta; C. (2011). Head movements; facial expressions and feedback in Danish first en- counters interactions: a culture-specific analysis. In C. Stephanidis; editor; Universal Access in Human- Computer Interaction. Users Diversity. Proceedings of 6th International Conference; UAHCI 2011; Held as Part of HCI International 2011; pp. 583–590; Orlando; FL; USA; July. Springer.
Paggio; P.; Ahlse´n; E.; Allwood; J.; Jokinen; K. & Navarretta; C. (2010). The NOMCO multimodal Nordic resource - goals and characteristics. In Pro- ceedings of LREC 2010; pp. 2968–2973; Malta; May 17-23.
Peirce.; C. S. (1931). Collected Papers of Charles Sanders Peirce; 1931-1958; 8 vols. Harvard Uni- versity Press; Cambridge;MA.
Paggio; P. & Navarretta; C. (2012). Head movements; facial expressions and feedback in conversations - empirical evidence from danish multimodal data. Journal on Multimodal User Interfaces - Special Issue on Multimodal Corpora.
Sacks; H.; Schegloff; E. & Jefferson; G.. (1974). A sim- plest systematics for the organization of turn-taking for conversation. Language; 50(4); 696–735.
Schegloff; E. (2000). Overlapping talk and the orga- nization of turn-taking for conversation. Language in Society; 29:1–63.
