Expressivity of Emergent Language is a Trade-off between Contextual Complexity and Unpredictability

  • 2021-06-07 21:57:11
  • Shangmin Guo, Yi Ren, Kory Mathewson, Simon Kirby, Stefano V. Albrecht, Kenny Smith
  • 0

Abstract

Researchers are now using deep learning models to explore the emergence oflanguage in various language games, where simulated agents interact and developan emergent language to solve a task. Although it is quite intuitive thatdifferent types of language games posing different communicative challengesmight require emergent languages which encode different levels of information,there is no existing work exploring the expressivity of the emergent languages.In this work, we propose a definition of partial order between expressivitybased on the generalisation performance across different language games. Wealso validate the hypothesis that expressivity of emergent languages is atrade-off between the complexity and unpredictability of the context thoselanguages are used in. Our second novel contribution is introducing contrastiveloss into the implementation of referential games. We show that using ourcontrastive loss alleviates the collapse of message types seen using standardreferential loss functions.

 

Quick Read (beta)

loading the full paper ...