Learning to Reason with Third-Order Tensor Products

  • 2018-11-29 13:50:58
  • Imanol Schlag, J├╝rgen Schmidhuber
  • 24

Abstract

We combine Recurrent Neural Networks with Tensor Product Representations tolearn combinatorial representations of sequential data. This improves symbolicinterpretation and systematic generalisation. Our architecture is trainedend-to-end through gradient descent on a variety of simple natural languagereasoning tasks, significantly outperforming the latest state-of-the-art modelsin single-task and all-tasks settings. We also augment a subset of the datasuch that training and test data exhibit large systematic differences and showthat our approach generalises better than the previous state-of-the-art.

 

Introduction (beta)

None

 

Conclusion (beta)

None