Conference article

Exploring Sounded and Silent Laughter in Multiparty Task-based and Social Interaction - Audio; Video and Biometric Signals

Emer Gilmartin
Speech Communications Lab, Trinity College Dublin, Ireland

Shannon Hennig
PAVIS Group, Istituto Italiano di Tecnologia, Italy

Ryad Chellali
PAVIS Group, Istituto Italiano di Tecnologia, Italy

Nick Campbell
Speech Communications Lab, Trinity College Dublin, Ireland

Download article

Published in: Proceedings from the 1st European Symposium on Multimodal Communication University of Malta; Valletta; October 17-18; 2013

Linköping Electronic Conference Proceedings 101:3, p. 17-27

NEALT Proceedings Series 21:3, p. 17-27

Show more +

Published: 2014-06-24

ISBN: 978-91-7519-266-6

ISSN: 1650-3686 (print), 1650-3740 (online)

Abstract

We report on our explorations of laughter in multiparty spoken interaction. Laughter is universally observed in human interaction. It is multimodal in nature: a stereotyped exhalation from the mouth in conjunction with rhythmic head and body movement. Predominantly occurring in company rather than solo; it is believed to aid social bonding. Spoken interaction is widely studied through corpus analysis; often concentrating on ‘taskbased’ interactions such as information gap activities and real or staged business meetings. Task-based interactions rely heavily on verbal information exchange while the immediate task in natural conversation is social bonding. We investigate laughter in situ in task-based and social interaction; using corpora of non-scripted (spontaneous) multiparty interaction: the task-oriented AMI meetings corpus; and the conversational TableTalk; and D-ANS corpora. We outline extension of previous work on laughter and topic change; describe the collection of natural social spoken interaction in the DANS corpus including audio-visual and biosignals; and describe an annotation experiment on multimodal aspects of laughter. We discuss the results and signal current and future research directions.

Keywords

Laughter; conversation analysis; social conversation; multimodal communication

References

Allen; James F.; Donna K. Byron; Myroslava Dzikovska; George Ferguson; Lucian Galescu; and Amanda Stent. 2001. ‘Toward Conversational Human- Computer Interaction’. AI Magazine 22 (4): 27.

Allwood; J.; L. Cerrato; L. Dybkær; and P. Paggio. 2004. ‘The MUMIN Multimodal Coding Scheme’. In Proc. Workshop on Multimodal Corpora and Annotation.

Allwood; Jens; Maria Björnberg; Leif Grönqvist; Elisabeth Ahlsén; and Cajsa Ottesjö. 2000. ‘The Spoken Language Corpus at the Department of Linguistics; Göteborg University’. In FQS–Forum Qualitative Social Research. Vol. 1.

Anderson; A.H.; M. Bader; E.G. Bard; E. Boyle; G. Doherty; S. Garrod; S. Isard; et al. 1991. ‘The HCRC Map Task Corpus’. Language and Speech 34 (4): 351–66.

Bachorowski; J. A; and M. J Owren. 1995. ‘Vocal Expression of Emotion: Acoustic Properties of Speech Are Associated with Emotional Intensity and Context’. Psychological Science; 219–24.

Baker; Rachel; and Valerie Hazan. 2010. ‘LUCID: A Corpus of Spontaneous and Read Clear Speech in British English’. In Proceedings of the DiSS-LPSS Joint Workshop 2010.

Bakhtine; Mikhail Mikhaïlovitch. 1986. Speech Genres and Other Late Essays. 8. University of Texas Press.

Black; Donald W. 1984. ‘Laughter’. JAMA: The Journal of the American Medical Association 252 (21): 2995–98.

BNC-Consortium. 2000. ‘British National Corpus’. URL Http://www. Hcu. Ox. Ac. uk/BNC.

Boersma; Paul; and David Weenink. 2010. Praat: Doing Phonetics by Computer [Computer Program]; Version 5.1. 44.

Bonin; Francesca; Nick Campbell; and Carl Vogel. 2012. ‘Laughter and Topic Changes: Temporal Distribution and Information Flow’. In Cognitive
Infocommunications (CogInfoCom); 2012 IEEE 3rd International Conference on; 53–58.

Burling; R. 2007. The Talking Ape: How Language Evolved. Vol. 5. Oxford University Press; USA.

Campbell; Nick. 2004. ‘Speech & Expression; the Value of a Longitudinal Corpus.’ In Proc. LREC 2004.

Campbell; Nick. 2007. ‘Approaches to Conversational Speech Rhythm: Speech Activity in Two-Person Telephone Dialogues’. In Proc XVIth International Congress of the Phonetic Sciences; Saarbrucken; Germany; 343–48.

Cheepen; Christine. 1988. The Predictability of Informal Conversation. Pinter London. Dawson; Michael E.; Anne M. Schell; and Diane L. Filion. 2007. ‘The Electrodermal System’. Handbook of Psychophysiology; 159.

DuBois; John W.; W. L. Chafe; C. Meyer; and S. A. Thompson. 2000. Santa Barbara Corpus of Spoken American English. CD-ROM. Philadelphia: Linguistic Data Consortium.

Dunbar; R. 1998. Grooming; Gossip; and the Evolution of Language. Harvard Univ Press.

Edlund; Jens; Jonas Beskow; Kjell Elenius; Kahl Hellmer; Sofia Strömbergsson; and David House. 2010. ‘Spontal: A Swedish Spontaneous Dialogue Corpus of Audio; Video and Motion Capture.’ In LREC. Gilmartin; Emer; Francesca Bonin; Carl Vogel; and Nick Campbell. 2013. ‘Laugher and Topic Transition in Multiparty Conversation’. In Proceedings of the SIGDIAL 2013 Conference; 304–8. Metz; France: Association for Computational Linguistics.

Gilmartin; Emer; and Nick Campbell. 2014. ‘More Than Just Words: Building a Chatty Robot’. In Natural Interaction with Robots; Knowbots and Smartphones; Glenn; Phillip J. 2003. Laughter in Interaction. Cambridge University Press Cambridge.

Godfrey; John J.; Edward C. Holliman; and Jane McDaniel. 1992. ‘SWITCHBOARD: Telephone Speech Corpus for Research and Development’. In Acoustics; Speech; and Signal Processing; 1992. ICASSP- 92.; 1992 IEEE International Conference on; 1:517–20.

Greenbaum; Sidney. 1991. ‘ICE: The International Corpus of English’. English Today 28 (7.4): 3–7.

Hennig; Shannon; Ryad Chellali; and Nick Campbell. 2014. ‘The D-ANS Corpus: The Dublin-Autonomous Nervous System Corpus of Biosignal and Multimodal Recordings of Conversational Speech.’ In Proc. LREC 2014; Reykjavik; Iceland.

Holt; Elizabeth. 2010. ‘The Last Laugh: Shared Laughter and Topic Termination’. Journal of Pragmatics 42 (6): 1513–25.

Janin; Adam; Don Baron; Jane Edwards; Dan Ellis; David Gelbart; Nelson Morgan; Barbara Peskin; Thilo Pfau; Elizabeth Shriberg; and Andreas Stolcke. 2003. ‘The ICSI Meeting Corpus’. In Acoustics; Speech; and Signal Processing; 2003. Proceedings.(ICASSP’03). 2003 IEEE International Conference on; 1:I–364.

Jokinen; Kristiina. 2009. ‘Gaze and Gesture Activity in Communication’. In Universal Access in Human-Computer Interaction. Intelligent and Ubiquitous Interaction Environments; 537–46. Springer.

Kahneman; D. 2011. Thinking; Fast and Slow. Farrar Straus & Giroux.

Laver; John. 1975. ‘Communicative Functions of Phatic Communion’. Organization of Behavior in Face-to-Face Interaction; 215–38.

Lemke; Jay L. 2012. ‘Analyzing Verbal Data: Principles; Methods; and Problems’. In Second International Handbook of Science Education; 1471–84. Springer.

Little; David. 2006. ‘The Common European Framework of Reference for Languages: Content; Purpose; Origin; Reception and Impact’. Language Teaching 39 (03): 167–90.

Malinowski; B. 1923. ‘The Problem of Meaning in Primitive Languages’. Supplementary in the Meaning of Meaning; 1–84.

McCowan; Iain; Jean Carletta; W. Kraaij; S. Ashby; S. Bourban; M. Flynn; M. Guillemot; T. Hain; J. Kadlec; and V. Karaiskos. 2005. ‘The AMI Meeting Corpus’. In Proceedings of the 5th International Conference on Methods and Techniques in Behavioral Research. Vol. 88.

Mehu; Marc; and Robin IM Dunbar. 2008. ‘Relationship between Smiling and Laughter in Humans (Homo Sapiens): Testing the Power Asymmetry Hypothesis’. Folia Primatologica 79 (5): 269–80.

Paggio; Patrizia; Jens Allwood; Elisabeth Ahlsén; and Kristiina Jokinen. 2010. NOMCO Multimodal Nordic Resource–goals and Characteristics’.

Poh; M. Z.; N. C. Swenson; and R. W. Picard. 2010. ‘A Wearable Sensor for Unobtrusive; Long-Term Assessment of Electrodermal Activity’. Biomedical Engineering; IEEE Transactions on 57 (5): 1243–52.

Provine; Robert R. 1993. ‘Laughter Punctuates Speech: Linguistic; Social and Gender Contexts of Laughter’. Ethology 95 (4): 291–98.

Provine; Robert R.. 2004. ‘Laughing; Tickling; and the Evolution of Speech and Self’. Current Directions in Psychological Science 13 (6): 215–18.

Sacks; H.; E.A. Schegloff; and G. Jefferson. 1974. ‘A Simplest Systematics for the Organization of Turn-Taking for Conversation’. Language; 696–735.

Scherer; Stefan; Friedhelm Schwenker; Nick Campbell; and Günther Palm. 2009. ‘Multimodal Laughter Detection in Natural Discourses’. In Human Centered Robot Systems; 111–20. Springer.

Schneider; Klaus P. 1988. Small Talk: Analysing Phatic Discourse. Vol. 1. Hitzeroth Marburg.

Shi; Yu; Natalie Ruiz; Ronnie Taib; Eric Choi; and Fang Chen. 2007. ‘Galvanic Skin Response (GSR) as an Index of Cognitive Load’. In CHI’07 Extended Abstracts on Human Factors in Computing Systems; 2651–56.

Thornbury; Scott; and Diana Slade. 2006. Conversation: From Description to Pedagogy. Cambridge University Press.

Trouvain; Jürgen. 2003. ‘Segmenting Phonetic Units in Laughter’. In Proceedings of the 15th International Congress of Phonetic Sciences. Barcelona: Universitat Autònoma de Barcelona; 2793–96.

Truong; Khiet P.; and Jurgen Trouvain. 2012. ‘Laughter Annotations in Conversational Speech Corpora-Possibilities and Limitations for Phonetic Analysis’. Proceedings of the 4th International Worskhop on Corpora for Research on Emotion Sentiment and Social Signals; 20–24.

Van Engen; Kristin J.; Melissa Baese-Berk; Rachel E. Baker; Arim Choi; Midam Kim; and Ann R. Bradlow. 2010. ‘The Wildcat Corpus of Native-and Foreign-
Accented English: Communicative Efficiency across Conversational Dyads with Varying Language Alignment Profiles’. Language and Speech 53 (4): 510–40.

Vinciarelli; Alessandro; Hugues Salamin; Anna Polychroniou; Gelareh Mohammadi; and Antonio Origlia. 2012. ‘From Nonverbal Cues to Perception: Personality and Social Attractiveness’. In Cognitive Behavioural Systems; 60–72. Springer.

Wittenburg; Peter; Hennie Brugman; Albert Russel; Alex Klassmann; and Han Sloetjes. 2006. ‘Elan: A Professional Framework for Multimodality Research’. In Proceedings of LREC. Vol. 2006.

Citations in Crossref