Abstract
Language models are often trained on text alone, without additionalgrounding. There is debate as to how much of natural language semantics can beinferred from such a procedure. We prove that entailment judgments betweensentences can be extracted from an ideal language model that has perfectlylearned its target distribution, assuming the training sentences are generatedby Gricean agents, i.e., agents who follow fundamental principles ofcommunication from the linguistic theory of pragmatics. We also show entailmentjudgments can be decoded from the predictions of a language model trained onsuch Gricean data. Our results reveal a pathway for understanding the semanticinformation encoded in unlabeled linguistic data and a potential framework forextracting semantics from language models.