Emergence of Machine Language: Towards Symbolic Intelligence with Neural Networks

  • 2022-01-14 14:54:58
  • Yuqi Wang, Xu-Yao Zhang, Cheng-Lin Liu, Zhaoxiang Zhang
  • 2

Abstract

Representation is a core issue in artificial intelligence. Humans usediscrete language to communicate and learn from each other, while machines usecontinuous features (like vector, matrix, or tensor in deep neural networks) torepresent cognitive patterns. Discrete symbols are low-dimensional, decoupled,and have strong reasoning ability, while continuous features arehigh-dimensional, coupled, and have incredible abstracting capabilities. Inrecent years, deep learning has developed the idea of continuous representationto the extreme, using millions of parameters to achieve high accuracies.Although this is reasonable from the statistical perspective, it has othermajor problems like lacking interpretability, poor generalization, and is easyto be attacked. Since both paradigms have strengths and weaknesses, a betterchoice is to seek reconciliation. In this paper, we make an initial attempttowards this direction. Specifically, we propose to combine symbolism andconnectionism principles by using neural networks to derive a discreterepresentation. This process is highly similar to human language, which is anatural combination of discrete symbols and neural systems, where the brainprocesses continuous signals and represents intelligence via discrete language.To mimic this functionality, we denote our approach as machine language. Bydesigning an interactive environment and task, we demonstrated that machinescould generate a spontaneous, flexible, and semantic language throughcooperation. Moreover, through experiments we show that discrete languagerepresentation has several advantages compared with continuous featurerepresentation, from the aspects of interpretability, generalization, androbustness.

 

Quick Read (beta)

loading the full paper ...