Abstract
Language is crucial for human intelligence, but what exactly is its role? Wetake language to be a part of a system for understanding and communicatingabout situations. The human ability to understand and communicate aboutsituations emerges gradually from experience and depends on domain-generalprinciples of biological neural networks: connection-based learning,distributed representation, and context-sensitive, mutual constraintsatisfaction-based processing. Current artificial language processing systemsrely on the same domain general principles, embodied in artificial neuralnetworks. Indeed, recent progress in this field depends on \emph{query-basedattention}, which extends the ability of these systems to exploit context andhas contributed to remarkable breakthroughs. Nevertheless, most current modelsfocus exclusively on language-internal tasks, limiting their ability to performtasks that depend on understanding situations. These systems also lack memoryfor the contents of prior situations outside of a fixed contextual span. Wedescribe the organization of the brain's distributed understanding system,which includes a fast learning system that addresses the memory problem. Wesketch a framework for future models of understanding drawing equally oncognitive neuroscience and artificial intelligence and exploiting query-basedattention. We highlight relevant current directions and consider furtherdevelopments needed to fully capture human-level language understanding in acomputational system.