Abstract

This survey discusses how recent developments in multimodal processingfacilitate conceptual grounding of language. We categorize the information flowin multimodal processing with respect to cognitive models of human informationprocessing and analyze different methods for combining multimodalrepresentations. Based on this methodological inventory, we discuss the benefitof multimodal grounding for a variety of language processing tasks and thechallenges that arise. We particularly focus on multimodal grounding of verbswhich play a crucial role for the compositional power of language.

 

Quick Read (beta)