Conference article

Investigating Non-Verbal Behaviors Conveying Interpersonal Stances

Mathieu Chollet
Institut Telecom, Telecom Paristech, CNRS-LTCI, France

Magalie Ochs
CNRS LTCI, Telecom ParisTech, France

Catherine Pelachaud
CNRS LTCI, Telecom ParisTech, France

Download article

Published in: Proceedings from the 1st European Symposium on Multimodal Communication University of Malta; Valletta; October 17-18; 2013

Linköping Electronic Conference Proceedings 101:2, p. 7-15

NEALT Proceedings Series 21:2, p. 7-15

Show more +

Published: 2014-06-24

ISBN: 978-91-7519-266-6

ISSN: 1650-3686 (print), 1650-3740 (online)

Abstract

Interpersonal stances are expressed by non-verbal behaviors on a variety of different modalities. The perception of these behaviors is influenced by the context of the interaction; how they are sequenced with other behaviors from the same person and behaviors from other interactants. In this paper; we introduce a framework considering the expressions of stances on different layers during an interaction. This framework enables one to reason on the nonverbal signals that an Embodied Conversational Agent should express to convey different stances. To identify more precisely humans’ non-verbal signals conveying dominance and friendliness attitudes; we propose in this paper a methodology to automatically extract the sequences of non-verbal signals conveying stances. The methodology is illustrated on an annotated corpus of job interviews.

Keywords

Interpersonal stance; Non-verbal behaviors; Sequence mining

References

R. Agrawal and R. Srikant. 1994. Fast algorithms for mining association rules in large databases. In Proceedings of the 20th International Conference on Very Large Data Bases; VLDB ’94; pages 487–499; San Francisco; CA; USA. Morgan Kaufmann Publishers Inc.

J. Allwood; S. Kopp; K. Grammer; E. Ahlsen; E. Oberzaucher; and M Koppensteiner. 2007. The analysis of embodied communicative feedback in multimodal corpora: a prerequisite for behavior simulation. Language Resources and Evaluation; 41:255–272.

M. Argyle. 1988. Bodily Communication. University paperbacks. Methuen.

R. Bakeman and V. Quera. 2011. Sequential Analysis and Observational Methods for the Behavioral Sciences. Cambridge University Press.

D. Ballin; M. Gillies; and B. Crabtree. 2004. A framework for interpersonal attitude and non-verbal communication in improvisational visual media production. In 1st European Conference on Visual Media Production.

E. Bevacqua; E. Sevin; S. J. Hyniewska; and C. Pelachaud. 2012. A listener model: introducing personality traits. Journal on Multimodal User Interfaces; 6(1-2):27–38.

T. W. Bickmore and R. W. Picard. 2005. Establishing and maintaining long-term human-computer relationships. ACM Trans. Comput.-Hum. Interact.; 12(2):293–327; June.

P. Boersma and D. Weenink. 2001. Praat; a system for doing phonetics by computer. Glot International; 5(9/10):341–345.

J. K. Burgoon; D. B. Buller; J. L. Hale; and M. A. de Turck. 1984. Relational Messages Associated with Nonverbal Behaviors. Human Communication Research; 10(3):351–378.

A. Cafaro; H. H. Vilhjálmsson; T. Bickmore; D. Heylen; K. R. Jóhannsdóttir; and G. S. Valgarðsson. 2012. First impressions: users’ judgments of virtual agents’ personality and interpersonal attitude in first encounters. In Proceedings of the 12th international conference on Intelligent Virtual Agents; IVA’12; pages 67–80; Berlin; Heidelberg. Springer-Verlag.

D. R. Carney; J. A. Hall; and L. S. LeBeau. 2005. Beliefs about the nonverbal expression of social power. Journal of Nonverbal Behavior; 29(2):105–123.

M. Chollet; M. Ochs; and C. Pelachaud. 2012. Interpersonal stance recognition using non-verbal signals on several time windows. In Workshop Affect; Compagnon Artificiel; Interaction.

M. Chollet; M. Ochs; and C. Pelachaud. 2013. A multimodal corpus approach to the design of virtual recruiters. In Workshop Multimodal Corpora; Intelligent Virtual Agents; pages 36–41.

R. Cowie and G. McKeown. 2010. Statistical analysis of data from initial labelled database and recommendations for an economical coding scheme.

R. Cowie; C. Cox; J.-C. Martin; A. Batliner; D. Heylen; and K. Karpouzis. 2011. Issues in Data Labelling. Springer-Verlag Berlin Heidelberg.

P. Ekman and V. Friesen. 1977. Manual for the Facial Action Coding System. Palo Alto: Consulting Psychologists Press.

S. Escalera; O. Pujol; P. Radeva; J. Vitria; and M. Anguera. 2010. Automatic detection of dominance and expected interest. EURASIP Journal on Advances in Signal Processing; 2010(1):12.

P. G. Ferreira and P. J. Azevedo. 2005. Protein sequence classification through relevant sequence mining and bayes classifiers. Progress in Artificial Intelligence; 3808:236–247.

D. Keltner. 1995. Signs of appeasement: Evidence for the distinct displays of embarrassment; amusement; and shame. Journal of Personality and Social Psychology; 68:441–454.

M. LaFrance. 1982. Posture mirroring and rapport. In M. Davis; editor; Interaction Rhythms: Periodicity in Communicative Behavior; pages 279–299. New York: Human Sciences Press.

J. Lee and S. Marsella. 2011. Modeling side participants and bystanders: The importance of being a laugh track. In Proceedings of the 10th International Conference on Intelligent Virtual Agents; IVA’11; pages 240–247; Berlin; Heidelberg. Springer-Verlag.

M. S. Magnusson. 2000. Discovering hidden time patterns in behavior: T-patterns and their detection. Behavior Research Methods; Instruments; Computers; 32:93–110.

H. P. Martínez and G. N. Yannakakis. 2011. Mining multimodal sequential patterns: a case study on affect detection. In Proceedings of the 13th international conference on multimodal interfaces; ICMI ’11; pages 3–10; New York; NY; USA. ACM.

A. Metallinou and S. Narayanan. 2013. Annotation and processing of continuous emotional attributes: Challenges and opportunities. In Automatic Face and Gesture Recognition; pages 1–8.

K. Prepin; M. Ochs; and C. Pelachaud. 2013. Beyond backchannels: co-construction of dyadic stancce by reciprocal reinforcement of smiles between virtual agents. In International Conference CogSci (Annual Conference of the Cognitive Science Society).

B. Ravenet; M. Ochs; and C. Pelachaud. 2013. From a user-created corpus of virtual agent’s non-verbal behaviour to a computational model of interpersonal attitudes. In International Conference on Intelligent Virtual Agent (IVA2013).

K. R. Scherer. 2005. What are emotions? and how can they be measured? Social Science Information; 44:695–729.

R. Srikant and R. Agrawal. 1996. Mining sequential patterns: Generalizations and performance improvements. Advances in Database Technology; 1057:1– 17.

P. Tan; M. Steinbach; and V. Kumar. 2005. Introduction to Data Mining;(First Edition). Addison-Wesley Longman Publishing Co.; Inc.; Boston; MA; USA.

S. With and W. S. Kaiser. 2011. Sequential patterning of facial actions in the production and perception of emotional expressions. Swiss Journal of Psychology; 70(4):241–252.

P. Wittenburg; H. Brugman; A. Russel; A. Klassmann; and H. Sloetjes. 2006. Elan: a professional framework for multimodality research. In In Proceedings of Language Resources and Evaluation Conference (LREC).

Citations in Crossref