Statistical graphs are tools for multimodal communication in daily life settings. For visually impaired people, haptic interfaces provide perceptual access to the information provided by the graph. Haptic comprehension can be facilitated by audio and verbal assistance. We investigate the circumstances un- der which verbal assistance facilitates haptic comprehension of graphs. For this, we focus on cases where unaided haptic graph comprehension has limitations, such as when describing a global (rather than a local) maximum. In an experiment that employed a joint activity setting, we observed two major factors for facilitating haptic-graph comprehension by providing verbal assistance: First, the results re- vealed significant impact of the explorer as a dialogue initiator during the course of haptic exploration. Second, verbal assistance led to more successful graph comprehension when it was enriched by modifi- ers.
Iyad Abu Doush, Enrico Pontelli, Tran C. Son, Dominic Simon, and Ou Ma. 2010. Multimodal presentation of two-dimensional charts: an investigation using Open Office XML and Microsoft Excel. ACM Transactions on Accessible Computing, 3(2), Article 8.
Cengiz Acartürk. 2010. Multimodal comprehension of graph-text constellations: An information processing perspective. Doctoral dissertation, University of Hamburg, Hamburg.
Cengiz Acartürk, Özge Alaçam, and Christopher Habel. 2014. Developing a verbal assistance system for line graph comprehension. In A. Marcus (Ed.): Design, User Experience and Usability (DUXU/HCII 2014), Part
II, LNCS 8518. (pp. 373–382). Berlin: Springer-Verlag.
Cengiz Acartürk, Özge Alaçam, and Christopher Habel. 2015. Haptic exploration patterns in virtual line-graph comprehension. In A. Marcus (Ed.): Design, User Experience and Usability (DUXU/HCII 2015), Berlin: Springer-Verlag. (accepted)
Özge Alaçam, Christopher Habel, and Cengiz Acartürk. 2013a. Towards designing audio assistance for comprehending haptic graphs: A multimodal perspective. In Constantine Stephanidis and Margherita Antona (Eds.), Universal Access in Human-Computer Interaction. (UAHCI/HCII 2013), Part I, LNCS 8009. (pp. 409–418). Berlin: Springer-Verlag.
Özge Alaçam, Christopher Habel, and Cengiz Acartürk. 2013b. Investigation of haptic line-graph comprehension through co-production of gesture and language. Tilburg Gesture Research Meeting, Tilburg.
Özge Alaçam, Cengiz Acartürk, and Christopher Habel. 2014a. Referring expressions in eiscourse about haptic line graphs. In Proceedings of the 18th Workshop on the Semantics and Pragmatics of Dialogue. SemDial 2014 – DialWatt. Verena Rieser & Phillipe Muller (eds.). (pp. 7–16)
Özge Alaçam, Christopher Habel, and Cengiz Acartürk. 2014b. Perspective alignment during the course of verbally assisted haptic graph comprehension. In the online Proceedings of the RefNet Workshop 2014 on Psychological and Computational Models of Reference Comprehension and Production. Edinburgh, UK. 31
August 2014.
Domenic V. Cicchetti. 1994. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6(4), 284–290.
Herbert H. Clark and Susan E. Haviland. 1977. Comprehension and the given-new contract. In Roy O. Freedle (Ed.), Discourse Production and Comprehension. Hillsdale, NJ: Erlbaum.
Herbert H. Clark. 1996. Using Language. Cambridge: Cambridge University Press.
Robert Dale and Ehud Reiter. 1995. Computational interpretations of the Gricean maxims in the generation of referring expressions. Cognitive Science, 19(2), 233-263.
Simon Garrod and Martin J. Pickering. 2004. Why is conversation so easy? Trends in Cognitive Sciences, 8. (1), 8-11.
Christopher Habel and Cengiz Acartürk. 2007. On reciprocal improvement in multimodal generation: Coreference by text and information graphics. In Ielka van der Sluis, Mariet Theune, Ehud Reiter and Emiel Krahmer (Eds.), Proceedings of Workshop on Multimodal Output Generation, 69-80.
Christopher Habel, Özge Alaçam, and Cengiz Acartürk. 2013. Verbally assisted comprehension of haptic linegraphs: referring expressions in a collaborative activity. In Proceedings of the CogSci 2013 Workshop on Production of Referring Expressions, Berlin.
Mary Hegarty. 2011. The cognitive science of visual-spatial displays: implications for design. Topics in Cognitive Science, 3, 446–474.
Matthias Kerzel, Özge Alaçam, Christopher Habel, and Cengiz Acartürk. 2014. Producing verbal descriptions for haptic line-graph explorations. In Proceedings of the 18th Workshop on the Semantics and Pragmatics of Dialogue. SemDial 2014 – DialWatt. Verena Rieser & Phillipe Muller (eds.). (pp. 205 – 207).
Jack M. Loomis and Roberta L. Klatzky. 2008. Functional equivalence of spatial representations from vision, touch, and hearing: Relevance for sensory substitution. In John D. Rieser, Daniel H. Ashmead, Ford F. Ebner,& Anne L. Corn (eds). Blindness and brain plasticity in navigation and object perception, (pp. 155-184). New
York: Erlbaum.
Jack M. Loomis, Roberta L. Klatzky, and Susan J. Lederman. 1991. Similarity of tactual and visual picture recognition with limited field of view. Perception, 20, 167-177.
PRBO. Waterbird Census at Bolinas Lagoon, MarinCounty, CA. Public report by Wetlands Ecology Division, Point Reyes Bird Observatory (PRBO) Conservation Science. (2012) http://www.prbo.org/cms/366, retrieved on January 29, 2012.
Marc Schröder and Jürgen Trouvain. 2003. The German Text-to-Speech synthesis system MARY: A tool for research, development and teaching. International Journal of Speech Technology, 6, 365-377.
Barbara Tversky. 1999. What does drawing reveal about thinking?. In John S. Gero, Barbara Tversky (eds.) Visual and spatial reasoning in design (VR99), Key Centre of Design Computing and Cognition, University of Sydney.
Wai Yu and Stephen A. Brewster. 2003. Evaluation of multimodal graphs for blind people. Journal of Universal Access in the Information Society, 2, 105-124.
Haixia Zhao, Catherine Plaisant, Ben Schneiderman, and Jonathan Lazar. 2008. Data sonification for users with visual impairment: a case study with georeferenced data. ACM Transactions on Computer-Human Interaction (TOCHI), 15(1).
