Abstract
Does the effectiveness of neural language models derive entirely fromaccurate modeling of surface word co-occurrence statistics, or do these modelsrepresent and reason about the world they describe? In BART and T5 transformerlanguage models, we identify contextual word representations that function asmodels of entities and situations as they evolve throughout a discourse. Theseneural representations have functional similarities to linguistic models ofdynamic semantics: they support a linear readout of each entity's currentproperties and relations, and can be manipulated with predictable effects onlanguage generation. Our results indicate that prediction in pretrained neurallanguage models is supported, at least in part, by dynamic representations ofmeaning and implicit simulation of entity state, and that this behavior can belearned with only text as training data. Code and data are available athttps://github.com/belindal/state-probes .