Revisiting the poverty of the stimulus: hierarchical generalization without a hierarchical bias in recurrent neural networks

  • 2018-06-08 04:20:31
  • R. Thomas McCoy, Robert Frank, Tal Linzen
  • 0

Abstract

Syntactic rules in natural language typically need to make reference tohierarchical sentence structure. However, the simple examples that languagelearners receive are often equally compatible with linear rules. Childrenconsistently ignore these linear explanations and settle instead on the correcthierarchical one. This fact has motivated the proposal that the learner'shypothesis space is constrained to include only hierarchical rules. We examinethis proposal using recurrent neural networks (RNNs), which are not constrainedin such a way. We simulate the acquisition of question formation, ahierarchical transformation, in a fragment of English. We find that some RNNarchitectures tend to learn the hierarchical rule, suggesting that hierarchicalcues within the language, combined with the implicit architectural biasesinherent in certain RNNs, may be sufficient to induce hierarchicalgeneralizations. The likelihood of acquiring the hierarchical generalizationincreased when the language included an additional cue to hierarchy in the formof subject-verb agreement, underscoring the role of cues to hierarchy in thelearner's input.

 

Quick Read (beta)

loading the full paper ...