Abstract

We deploy the methods of controlled psycholinguistic experimentation to shedlight on the extent to which the behavior of neural network language modelsreflects incremental representations of syntactic state. To do so, we examinemodel behavior on artificial sentences containing a variety of syntacticallycomplex structures. We test four models: two publicly available LSTM sequencemodels of English (Jozefowicz et al., 2016; Gulordava et al., 2018) trained onlarge datasets; an RNNG (Dyer et al., 2016) trained on a small, parsed dataset;and an LSTM trained on the same small corpus as the RNNG. We find evidence thatthe LSTMs trained on large datasets represent syntactic state over large spansof text in a way that is comparable to the RNNG, while the LSTM trained on thesmall dataset does not or does so only weakly.

 

Introduction (beta)

None

 

Conclusion (beta)

None