Marie-Catherine de Marneffe
Linguistics Department, The Ohio State University, Columbus, OH, USA
Matias Grioni
Computer Science Department, The Ohio State University, Columbus, OH, USA
Jenna Kanerva
Turku NLP group, University of Turku, Finland
Filip Ginter
Turku NLP group, University of Turku, Finland
Download article
Published in: Proceedings of the Fourth International Conference on Dependency Linguistics (Depling 2017), September 18-20, 2017, Università di Pisa, Italy
Linköping Electronic Conference Proceedings 139:14, p. 108-115
Published: 2017-09-13
ISBN: 978-91-7685-467-9
ISSN: 1650-3686 (print), 1650-3740 (online)
A fundamental issue in annotation efforts is to ensure that the same phenomena within and across corpora are annotated consistently. To date, there has not been a clear and obvious way to ensure annotation consistency of dependency corpora. Here, we revisit the method of Boyd et al. (2008) to flag inconsistencies in dependency corpora, and evaluate it on three languages with varying degrees of morphology (English, French, and Finnish UD v2).We show that the method is very efficient in finding errors in the annotations. We also build an annotation tool, which we will make available, that helps to streamline the manual annotation required by the method.
