Abstract

Many complex generative systems use languages to create structured objects.We consider a model of random languages, defined by weighted context-freegrammars. As the distribution of grammar weights broadens, a transition isfound from a random phase, in which sentences are indistinguishable from noise,to an organized phase in which nontrivial information is carried. This marksthe emergence of deep structure in the language, and can be understood by acompetition between energy and entropy.

 

Quick Read (beta)