Abstract
We explore the emergence of intelligent behavior in artificial systems byinvestigating how the complexity of rule-based systems influences thecapabilities of models trained to predict these rules. Our study focuses onelementary cellular automata (ECA), simple yet powerful one-dimensional systemsthat generate behaviors ranging from trivial to highly complex. By trainingdistinct Large Language Models (LLMs) on different ECAs, we evaluated therelationship between the complexity of the rules' behavior and the intelligenceexhibited by the LLMs, as reflected in their performance on downstream tasks.Our findings reveal that rules with higher complexity lead to models exhibitinggreater intelligence, as demonstrated by their performance on reasoning andchess move prediction tasks. Both uniform and periodic systems, and often alsohighly chaotic systems, resulted in poorer downstream performance, highlightinga sweet spot of complexity conducive to intelligence. We conjecture thatintelligence arises from the ability to predict complexity and that creatingintelligence may require only exposure to complexity.