Abstract
Language models trained on billions of tokens have recently led tounprecedented results on many NLP tasks. This success raises the question ofwhether, in principle, a system can ever "understand" raw text without accessto some form of grounding. We formally investigate the abilities of ungroundedsystems to acquire meaning. Our analysis focuses on the role of "assertions":contexts within raw text that provide indirect clues about underlyingsemantics. We study whether assertions enable a system to emulaterepresentations preserving semantic relations like equivalence. We find thatassertions enable semantic emulation if all expressions in the language arereferentially transparent. However, if the language uses non-transparentpatterns like variable binding, we show that emulation can become anuncomputable problem. Finally, we discuss differences between our formal modeland natural language, exploring how our results generalize to a modal settingand other semantic relations. Together, our results suggest that assertions incode or language do not provide sufficient signal to fully emulate semanticrepresentations. We formalize ways in which ungrounded language models appearto be fundamentally limited in their ability to "understand".