Conference article

Towards fine-grained readability measures for self-directed language learning

Lisa Beinborn
Ubiquitous Knowledge Processing Lab (UKP-TUDA), Department of Computer Science, Technische Universität Darmstadt

Torsten Zesch
Ubiquitous Knowledge Processing Lab (UKP-TUDA), Department of Computer Science, Technische Universität Darmstadt

Iryna Gurevych
Ubiquitous Knowledge Processing Lab (UKP-TUDA), Department of Computer Science, Technische Universität Darmstadt

Download article

Published in: Proceedings of the SLTC 2012 workshop on NLP for CALL; Lund; 25th October; 2012

Linköping Electronic Conference Proceedings 80:2, p. 11-19

Show more +

Published: 2012-11-12


ISSN: 1650-3686 (print), 1650-3740 (online)


In this paper; we analyze existing readability measures regarding their applicability to self-directed language learning. We identify a set of dimensions for text complexity and focus on the lexical; syntactic; semantic; and discourse dimensions. We argue that for the purposes of self-directed language learning; the assessment according to the individual dimensions should be preferred over the overall readability prediction. Furthermore; due to the heterogeneity of the learners in such a setting; modeling the background knowledge of the learner becomes a critical step.


No keywords available


Ahmed Al-Hmouz; Jun Shen; Jun Yan; and Rami Al- Hmouz. 2010. Enhanced learner model for adaptive mobile learning. In 12th International Conference on Information Integration and Web-based Applications & Services - iiWAS ’10; pages 783–786; New York; USA. ACM Press.

Hend S. Al-Khalifa and Amany Al-Ajlan. 2010. Automatic Readability Measurements of the Arabic Text: An Exploratory Study. The Arabian Journal for Science and Engineering; 35(2C).

Sandra Aluisio; Lucia Specia; Caroline Gasperin; and Carolina Scarton. 2010. Readability assessment for text simplification. In Proceedings of the NAACL HLT 2010 Fifth Workshop on Innovative Use of NLP for Building Educational Applications; pages 1–9. Association for Computational Linguistics; June.

Taher Bahrani and Tam Shu Sim. 2012. Informal language learning setting: technology or social interaction? mThe Turkish Online Journal of Educational Technology; 11(2):142–149.

Rebekah George Benjamin. 2011. Reconstructing Readability: Recent Developments and Recommendations in the Analysis of Text Difficulty. Educational Psychology Review; 24(1):63–88; October.

Mery Coleman and TL Liau. 1975. A computer readability formula designed for machine scoring. Journal of Applied Psychology; 60(2):283–284.

Kevyn Collins-Thompson and Jamie Callan. 2005. Predicting Reading Difficulty with Statistical Language Models. Journal of the American Society for Information Science and Technology; 56(13):1448–1462.

Vivian J. Cook; John Long; and Steve McDonough. 1979. First and second language learning. The Mother Tongue and Other Languages in Education; pages 7– 22.

Edgar Dale and Jeanne S Chall. 1948. A formula for predicting readability: Instructions. Educational Research Bulletin.

Felice Dell’Orletta; Simonetta Montemagni; and Giulia Venturi. 2011. READ-IT: assessing readability of Italian texts with a view to text simplification. In Proceedings of the Second Workshop on Speech and Language Processing for Assistive Technologies; pages 73–83; July.

William H. DuBay. 2004. The Principles of Readability. Impact Information; pages 1–76.

Christiane Fellbaum. 1998. WordNet: An electronic database. MIT Press; Cambridge; MA.

Lijun Feng and Matt Huenerfauth. 2010. A Comparison of Features for Automatic Readability Assessment. In Proceedings of COLING 2010; pages 276–284; August.

Thomas Franc¸ois and C´edrick Fairon. 2012. An AI readability formula for French as a foreign language.
In Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning; pages 466–477; July.

Edgar Fry. 1977. Fry’s readability graph: Clarifications; validity; and extension to level 17. Journal of Reading.

Arthur C. Graesser and Danielle McNamara. 2004. Coh- Metrix: Analysis of text on cohesion and language. Behavior Research Methods; 36(2).

Barbara J. Grosz; Scott Weinstein; and Aravind K. Joshi. 1995. Centering: A framework for modeling the local coherence of discourse. Computational linguistics; 21(2):203–225.

Robert Gunning. 1969. The Fog Index after twenty years. Journal of Business Communication; 6(2):3– 13.

Sven Hartrumpf. 2003. Hybrid Disambiguation in Natural Language Analysis. Der Andere Verlag; Osnabr ¨uck; Germany.

Michael J Heilman; Kevyn Collins-Thompson; Jamie Callan; and Maxine Eskenazi. 2007. Combining Lexical and Grammatical Features to Improve Readability Measures for First and Second Language Texts. In Proceedings of NAACL-HLT; pages 460–467.

Michael Heilman; Kevyn Collins-Thompson; and Maxine Eskenazi. 2008. An Analysis of Statistical Models and Features for Reading Difficulty Prediction. In Proceedings of the Third Workshop on Innovative Use of NLP for Building Educational Applications - EANL ’08; pages 71–79; Morristown; NJ; USA; June.

Filip Karel and Ji´i Klema. 2006. Adaptivity in e-learning. Current Developments in Technology- Assisted Education; 1:260–264.

John P. Kincaid; Robert P. Fishburne Jr; Richard L. Rogers; and Brad S. Chissom. 1975. Derivation of New Readability Formulas (Automated Readability Index; Fog Count and Flesch Reading Ease Formula) for Navy Enlisted Personnel. Technical report; DTIC Document.

Patrik Larsson. 2006. Classification into Readability Levels Implementation and Evaluation. Master’s thesis; Uppsala University; Sweden.

G. Harry McLaughlin. 1969. SMOG grading: A new readability formula. Journal of reading; 12(8):639– 646.

Terence Odlin. 1989. Language Transfer: Cross- Linguistic Influence in Language Learning. Cambridge University Press.

Emily Pitler and Ani Nenkova. 2008. Revisiting Readability: A Unified Framework for Predicting Text Quality. In Proceedings of the Conference on Empirical Methods in Natural Language Processing; pages 186–195. Association for Computational Linguistics.

Satoshi Sato; Suguru Matsuyoshi; and Yohsuke Kondoh. 2008. Automatic Assessment of Japanese Text Readability Based on a Textbook Corpus. In 6th International Conference on Language Resources and Evaluation (LREC); Marrakech; Morocco.

Richard Schmidt. 1995. Consciousness and foreign language learning: A tutorial on the role of attention and awareness in learning. Attention and Awareness in Foreign Language Learning; pages 1–63.

Sarah E Schwarm and Mari Ostendorf. 2005. Reading Level Assessment Using Support Vector Machines and Statistical Language Models. In Proceedings of the 43rd Annual Meeting of the ACL; pages 523–530; June.

Bengt Sigurd; Mats Eeg-Olofsson; and Joost van Weijer. 2004. Word length; sentence length and frequency - Zipf revisited. Studia Linguistica; 58(1):37–52; April.

E. A. Smith and R.J. Senter. 1967. Automated readability index. Cincinnati University Ohio.

Kumiko Tanaka-Ishii; Satoshi Tezuka; and Hiroshi Terada. 2010. Sorting Texts by Readability. Computational Linguistics; 36(2):203–227; June.

Sara Tonelli; Ke Tran Manh; and Emanuele Pianta. 2012. Making Readability Indices Readable. In Proceedings of NAACL-HLT: Workshop on Predicting and Improving Text Readability for target reader populations; pages 40–48.

Sowmya Vajjala and Detmar Meurers. 2012. On Improving the Accuracy of Readability Classification using Insights from Second Language Acquisition. In Proceedings of the 7th Workshop on Innovative Use of NLP for Building Educational Applications (BEA7); pages 163–173.

Maria Virvou and Christos Troussas. 2011. Web-based student modeling for learning multiple languages. In International Conference on Information Society (i- Society); pages 423–428.

Tim Vor der Br¨uck; Sven Hartrumpf; and Hermann Helbig. 2008. A Readability Checker with Supervised Learning using Deep Syntactic and Semantic Indicators. In 11th International Multiconference: Information Society-IS; pages 92–97.

Lev Vygotsky. 1978. Mind in society: The development of higher psychological processes. Harvard University Press.

Helmut Zobl. 1980. Developmental and transfer errors: their common bases and (possibly) differential effects on subsequent learning. TESOL Quarterly.

Citations in Crossref