Lexical information is an important feature in syntactic processing like part-ofspeech (POS) tagging and dependency parsing. However, there is no such information available for out-of-vocabulary (OOV) words, which causes many classification errors. We propose to replace OOV words with in-vocabulary words that are semantically similar according to distributional similar words computed from a large background corpus, as well as morphologically similar according to common suffixes. We show performance differences both for count-based and dense neural vector-based semantic models. Further, we discuss the interplay of POS and lexical information for dependency parsing and provide a detailed analysis and a discussion of results: while we observe significant improvements for count-based methods, neural vectors do not increase the overall accuracy.
Jacob Andreas and Dan Klein. 2014. How much do word embeddings encode about syntax? In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 822-827, Baltimore, Maryland.
Miguel Ballesteros and Joakim Nivre. 2016. Maltoptimizer: Fast and effective parser optimization. Natural Language Engineering, 22:187-213, 3.
Mohit Bansal, Kevin Gimpel, and Karen Livescu. 2014. Tailoring Continuous Word Representations for Dependency Parsing. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, ACL ’14, pages 809-815, Baltimore, MA, USA.
Chris Biemann and Martin Riedl. 2013. Text: Now in 2D! A Framework for Lexical Expansion with Contextual Similarity. Journal of Language Modelling, 1(1):55-95.
Bernd Bohnet. 2010. Very High Accuracy and Fast Dependency Parsing is Not a Contradiction. In Proceedings of the 23rd International Conference on Computational Linguistics, COLING ’10, pages 89-97, Beijing, China.
Peter F. Brown, Peter V. deSouza, Robert L. Mercer, Vincent J. Della Pietra, and Jenifer C. Lai. 1992. Class-based N-gram Models of Natural Language. Computational Linguistics, 18(4):467-479.
Wenliang Chen, Yue Zhang, and Min Zhang. 2014. Feature Embedding for Dependency Parsing. In Proceedings of the 25th International Conference on Computational Linguistics, COLING 2014, pages 816-826, Dublin, Ireland.
Stefan Evert. 2005. The Statistics of Word Cooccurrences: Word Pairs and Collocations. Ph.D. thesis, Institut f¨ur maschinelle Sprachverarbeitung, University of Stuttgart.
Rashmi Gangadharaiah, Ralf D. Brown, and Jaime Carbonell. 2010. Monolingual Distributional Profiles for Word Substitution in Machine Translation. In Proceedings of the 23rd International Conference on Computational Linguistics, COLING 2010, pages 320-328, Beijing, China.
Zellig Sabbetai Harris. 1951. Methods in Structural Linguistics. University of Chicago Press, Chicago.
Hen-Hsen Huang, Huan-Yuan Chen, Chang-Sheng Yu, Hsin-Hsi Chen, Po-Ching Lee, and Chun-Hsun Chen. 2014. Sentence Rephrasing for Parsing Sentences with OOV Words. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 26-31, Reykjavik, Iceland.
Ronald M. Kaplan and Joan Bresnan. 1982. Lexical-Functional Grammar: A Formal System for Grammatical Representation. In The Mental Representation of Grammatical Relations, pages 173-281. MIT Press, Cambridge, MA.
Terry Koo, Xavier Carreras, and Michael Collins. 2008. Simple Semi-supervised Dependency Parsing. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, ACL ’08, pages 595-603, Columbus, OH, USA.
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient Estimation of Word Representations in Vector Space. In Proceedings of the International Conference on Machine Learning, ICLR 2013, pages 1310-1318, Scottsdale, AZ, USA.
Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Yoav Goldberg, Jan Hajic, Christopher D. Manning, Ryan McDonald, Slav Petrov, Sampo Pyysalo, Natalia Silveira, Reut Tsarfaty, and Daniel Zeman. 2016. Universal dependencies v1: A multilingual treebank collection. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), Portoro?z, Slovenia.
Joakim Nivre. 2009. Non-projective dependency parsing in expected linear time. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, ACL’09, pages 351-359, Suntec, Singapore.
Robert Parker, David Graff, Junbo Kong, Ke Chen, and Kazuaki Maeda. 2011. English Gigaword Fifth Edition. Linguistic Data Consortium, Philadelphia. Matthias Richter, Uwe Quasthoff, Erla Hallsteinsdóttir, and Chris Biemann. 2006. Exploiting the Leipzig Corpora Collection. In Proceedings of the IS-LTC 2006, pages 68-73, Ljubljana, Slovenia.
Roland Schäfer and Felix Bildhauer. 2013. Web Corpus Construction. Synthesis Lectures on Human Language Technologies. Morgan and Claypool. Richard Socher, John Bauer, Christopher D. Manning, and Andrew Y. Ng. 2013. Parsing with Compositional
Vector Grammars. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, ACL ’13, pages 455-465, Sofia, Bulgaria.
Qin Iris Wang, Dale Schuurmans, and Dekang Lin. 2005. Strictly Lexical Dependency Parsing. In Proceedings of the Ninth International Workshop on Parsing Technology, Parsing ’05, pages 152-159, Vancouver, BC, Canada.
Jiajun Zhang, Feifei Zhai, and Chengqing Zong. 2012. Handling Unknown Words in Statistical Machine Translation from a New Perspective. In Proceedings of the 1st Conference on Natural Language Processing and Chinese Computing, NLP&CC ’12, pages 176-187, Beijing, China.
