Conference article

Enhancing Natural Language Understanding through Cross-Modal Interaction: Meaning Recovery from Acoustically Noisy Speech

Özge Alaçam
Department of Informatics, University of Hamburg, Hamburg, Germany

Download article

Published in: Proceedings of the 22nd Nordic Conference on Computational Linguistics (NoDaLiDa), September 30 - October 2, Turku, Finland

Linköping Electronic Conference Proceedings 167:28, p. 272--280

NEALT Proceedings Series 42:28, p. 272--280

Show more +

Published: 2019-10-02

ISBN: 978-91-7929-995-8

ISSN: 1650-3686 (print), 1650-3740 (online)


Cross-modality between vision and language is a key component for effective and ef?cient communication, and human language processing mechanism successfully integrates information from various modalities to extract the intended meaning. However, incomplete linguistic input, i.e. due to a noisy environment, is one of the challenges for a successful communication. In that case, an incompleteness in one channel can be compensated by information from another one. In this paper, by conducting visual-world paradigm, we investigated the dynamics between syntactically possible gap ?llers and the visual arrangements in incomplete German sentences and their effect on overall sentence interpretation.


text completion meaning recovery Negation situated language understanding


No references available

Citations in Crossref