Abstract
This paper pursues the insight that large language models (LLMs) trained togenerate code can vastly improve the effectiveness of mutation operatorsapplied to programs in genetic programming (GP). Because such LLMs benefit fromtraining data that includes sequential changes and modifications, they canapproximate likely changes that humans would make. To highlight the breadth ofimplications of such evolution through large models (ELM), in the mainexperiment ELM combined with MAP-Elites generates hundreds of thousands offunctional examples of Python programs that output working ambulating robots inthe Sodarace domain, which the original LLM had never seen in pre-training.These examples then help to bootstrap training a new conditional language modelthat can output the right walker for a particular terrain. The ability tobootstrap new models that can output appropriate artifacts for a given contextin a domain where zero training data was previously available carriesimplications for open-endedness, deep learning, and reinforcement learning.These implications are explored here in depth in the hope of inspiring newdirections of research now opened up by ELM.