Image-based Natural Language Understanding Using 2D Convolutional Neural Networks

  • 2018-11-06 10:46:24
  • Erinc Merdivan, Anastasios Vafeiadis, Dimitrios Kalatzis, Sten Hanke, Johannes Kropf, Konstantinos Votis, Dimitrios Giakoumis, Dimitrios Tzovaras, Liming Chen, Raouf Hamzaoui, Matthieu Geist
  • 0

Abstract

We propose a new approach to natural language understanding in which weconsider the input text as an image and apply 2D Convolutional Neural Networksto learn the local and global semantics of the sentences from the variationsofthe visual patterns of words. Our approach demonstrates that it is possibleto get semantically meaningful features from images with text without usingoptical character recognition and sequential processing pipelines, techniquesthat traditional Natural Language Understanding algorithms require. To validateour approach, we present results for two applications: text classification anddialog modeling. Using a 2D Convolutional Neural Network, we were able tooutperform the state-of-art accuracy results of non-Latin alphabet-based textclassification and achieved promising results for eight text classificationdatasets. Furthermore, our approach outperformed the memory networks when usingout of vocabulary entities fromtask 4 of the bAbI dialog dataset.

 

Quick Read (beta)

loading the full paper ...