Facing Nadine’s Speech. Multimodal Annotation of Emotion in Later Life

Catherine Bolly
CNRS, UMR 7023 SFL and Universitå catholique de Louvain, Belgium

Anaïs Thomas
University of Paris Ouest Nanterre, France

Ladda ner artikel

Ingår i: Proceedings of the 2nd European and the 5th Nordic Symposium on Multimodal Communication, August 6-8, 2014, Tartu, Estonia

Linköping Electronic Conference Proceedings 110:5, s. 25-35

Visa mer +

Publicerad: 2015-05-26

ISBN: 978-91-7519-074-7

ISSN: 1650-3686 (tryckt), 1650-3740 (online)


This pilot study aims at reconstructing the empathic profile of conversation participants from their inter- action in real-world settings. It addresses the question of how verbal and nonverbal modes converge in conveying information about the emotional and attitudinal behavior in everyday communication. In par- ticular, the empathic ability of older people is explored studying physiological patterning from nonver- bal resources, in relation to emotions expressed through the face. In addition, the IRI psychometric test of empathy provides the participant’s overall empathic profile. The data is taken from the CorpAGEst multimodal corpus, and focuses on the language of four healthy very old women who obtained a normal score on the MoCA cognitive test. Preliminary results indicate that, despite the highly idiosyncratic use of nonverbal resources, some inter- and intra-individual tendencies seem to emerge.


Inga nyckelord är tillgängliga


Allwood, J., Cerrato, L., Jokinen, K., Navarretta, C. and Paggio, P. 2007. The MUMIN coding scheme for the annotation of feedback, turn management and sequencing phenomena. Language Resources and Evaluation, 41(3-4), 273-287.

Bailey, Phoebe E. and Julie D. Henry. 2008. Growing less empathic with age: Disinhibition of the selfperspective. Journal of Gerontology: Psychological Sciences, 63B(4): 219–226.

Bolly, Catherine T. 2013. CorpAGEst. Multimodal corpus for the elderly’s language. Louvain-la-Neuve, Paris: Université catholique de Louvain (Valibel - Discours et Variation) and CNRS (UMR 7023 Structures Formelles du Langage), <http://corpagest.org>.

Brinton, Laurel J. 2008. The comment clause in English: Syntactic origins and pragmatic development. Cambridge : Cambrige University Press.

Colletta, Jean-Marc, Kunene, Ramona N., Venouil, Aurélie, Kaufmann, Virginie and Simon, Jean-Pascal. 2009. Multi-track annotation of child language and gestures. In Michael Kipp, Jean-Claude Martin, Patrizia Paggio and Dirk Heylen (eds.), Multimodal corpora (Lecture Notes in Computer Science 5509). Berlin, Heidelberg: Springer, 54–72.

Davis, Mark H. 1994. Empathy: A Social Psychological Approach. Boulder: Westview Press.

Douglas-Cowie, Ellen, Campbell, Nick, Cowie, Roddy, and Peter Roach. 2003. Emotional speech: Towards a new generation of databases. Speech Communication, 40: 33–60.

Du Bois, John W. and Elise Kärkkäinen 2012. Taking a stance on emotion: Affect, sequence, and intersubjectivity in dialogic interaction. Text & Talk, 32(4): 433–451.

Eisenberg, Nancy and Richard A. Fabes. 1990. Empathy: Conceptualization, measurement, and relation to prosocial behavior. Motivation and Emotion, 14(2): 131–149.

Eisenberg, Nancy, Shea, Cindy L., Carlo, Gustavo, and George P. Knight 2014. Empathy-related responding and cognition: A “chicken and the egg” dilemma. Handbook of moral behavior and development: Research, 2: 63–88.

Ekman, Paul. 1992. An argument for basic emotions. Cognition and Emotion, 6(3/4): 169–200.

Englebretson, Robert (ed.). 2007. Stancetaking in discourse: Subjectivity, evaluation, interaction (Pragmatics& Beyond New Series 164). Amsterdam, Philadelphia: John Benjamins Publishing.

Gendron, Maria, Lindquist, Kristen, Barsalou, Lawrence, and Lisa F. Barrett. 2012. Emotion words shape emotion percepts. Emotion, 12(2): 314–325.

Gilet, Anne-Laure, Mella, Nathalie, Studer, Joseph, Grühn, Daniel and Gisela Labouvie-Vief. 2013. Assessing dispositional empathy in adults: A French validation of the Interpersonal Reactivity Index (IRI). Canadian Journal of Behavioural Science / Revue canadienne des sciences du comportement, 45(1): 42–48.

Harwood, Jake. 2007. Understanding communication and aging: Developing knowledge and awareness. Thousand Oaks: Sage.

James, Lori E., Burke, Deborah M., Austin, Ayda and Erika Hulme. 1998. Production and perception of “verbosity” in younger and older adults. Psychology and Aging, 13(3): 353–367.

Kärkkäinen, Elise. 2006. Stance taking in conversation: From subjectivity to intersubjectivity. Text & Talk, 26(6): 699–731.

Kasper, Gabriele and Kenneth R. Rose. 2002. Pragmatic development in a second language. Mahwah: Blackwell (Also Language Learning : Supplement 1, 52).

Martin, Jean-Claude C., Niewiadomski, Radoslaw, Devillers, Laurence, Buisine, Stéphanie and Catherine Pelachaud. 2006. Multimodal complex emotions: Gesture expressivity and blended facial expressions. International Journal of Humanoid Robotics, 3(3): 269–291.

Müller, Cornelia, Jana Bressem and Silva H. Ladewig. 2013. Towards a grammar of gestures: A form-based view. In Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeill and Sedinha Teßendorf (eds.), Body – language – communication: An international handbook on multimodality in human interaction. Vol. 1. Berlin, Boston: Mouton de Gruyter, 707–733.

Nasreddine, Ziad S., Phillips, Natalie, Bédirian, Valérie, Charbonneau, Simon, Whitehead, Victor, Collin, Isabelle, Cummings, Jeffrey L. and Howard Chertkow. 2005. The Montreal Cognitive Assessment (MoCA): A brief screening tool for Mild Cognitive Impairment. Journal of the American Geriatrics Society, 53: 695–699, <http://www.mocatest.org/default.asp>.

Niedenthal, Paula M. 2007. Embodying emotion. Science, 316(5827): 1002–1005.

Niedenthal, Paula M., Barsalou, Lawrence W., Ric, François and Silvia Krauth-Gruber. 2005. Embodiment in the acquisition and use of emotion knowledge. In Lisa Feldman Barrett, Paula M. Niedenthal and Piotr Winkielman (eds.), Emotion and consciousness. New York: The Guilford Press, 21–50.

Plutchik, Robert. 1980. Theories of Emotion. Vol. 1: Emotion: Theory, Research, and Experience. New York: Academic.

Poggi, Isabella and Magno Caldognetto, Emanuela. 1996. A score for the analysis of gestures in multimodal communication. In L. Messing (ed.), Proceedings of the Workshop on the Integration of Gesture and Language in Speech, Applied Science and Engineering Laboratories, Newark and Wilmington, Delaware: 235–244.

Russell, James A., Jo-Anne Bachorowski and José-Miguel Fernández-Dols. 2003. Facial and vocal expressions of emotion. Annual review of psychology, 54(1): 329–349.

Wittenburg, Peter, Brugman, Hennie, Russel, Albert, Klassmann, Alex and Hans Sloetjes. 2006. Elan: A professional framework for multimodality research. In Proceedings of LREC 2006, Fifth International Conference on Language Resources and Evaluation. Max Planck Institute for Psycholinguistics, The Language Archive, Nijmegen, The Netherlands, <http://tla.mpi.nl/tools/tla-tools/elan/>.

Citeringar i Crossref