Article | Proceedings of the NoDaLiDa 2019 Workshop on Constraint Grammar - Methods, Tools and Applications, 30 September 2019, Turku, Finland | Constraint Grammar is a hand-crafted Transformer Linköping University Electronic Press Conference Proceedings
Göm menyn

Title:
Constraint Grammar is a hand-crafted Transformer
Author:
Anssi Yli-Jyrä: Helsinki Centre for Digital Humanities (HELDIG), P.O. Box 24, 00014 University of Helsinki, Finland
Download:
Full text (pdf)
Year:
2019
Conference:
Proceedings of the NoDaLiDa 2019 Workshop on Constraint Grammar - Methods, Tools and Applications, 30 September 2019, Turku, Finland
Issue:
168
Article no.:
009
Pages:
45-49
No. of pages:
4
Publication type:
Abstract and Fulltext
Published:
2019-12-03
ISBN:
978-91-7929-918-7
Series:
Linköping Electronic Conference Proceedings
ISSN (print):
1650-3686
ISSN (online):
1650-3740
Series:
NEALT Proceedings Series
Publisher:
Linköping University Electronic Press, Linköpings universitet


Export in BibTex, RIS or text

Deep neural networks (DNN) and linguistic rules are currently the opposite ends in the scale for NLP technologies. Until recently, it has not been known how to combine these technologies most effectively. Therefore, the technologies have been the object of almost disjoint research communities. In this presentation, I first recall that both Constraint Grammar (CG) and vanilla RNNs have finite-state properties. Then I relate CG to Google’s Transformer architecture (with two kinds of attention) and argue that there are significant similarities between these two seemingly unrelated architectures.

Keywords: constraint grammar, finite-state capacity, recurrent neural networks, self-attention, attention, rule conditions, Transformer

Proceedings of the NoDaLiDa 2019 Workshop on Constraint Grammar - Methods, Tools and Applications, 30 September 2019, Turku, Finland

Author:
Anssi Yli-Jyrä
Title:
Constraint Grammar is a hand-crafted Transformer
References:
No references available

Proceedings of the NoDaLiDa 2019 Workshop on Constraint Grammar - Methods, Tools and Applications, 30 September 2019, Turku, Finland

Author:
Anssi Yli-Jyrä
Title:
Constraint Grammar is a hand-crafted Transformer
Note: the following are taken directly from CrossRef
Citations:
No citations available at the moment


Responsible for this page: Peter Berkesand
Last updated: 2019-11-06