Abstract
This paper pursues the insight that language models naturally enable anintelligent variation operator similar in spirit to evolutionary crossover. Inparticular, language models of sufficient scale demonstrate in-contextlearning, i.e. they can learn from associations between a small number of inputpatterns to generate outputs incorporating such associations (also calledfew-shot prompting). This ability can be leveraged to form a simple butpowerful variation operator, i.e. to prompt a language model with a fewtext-based genotypes (such as code, plain-text sentences, or equations), and toparse its corresponding output as those genotypes' offspring. The promise ofsuch language model crossover (which is simple to implement and can leveragemany different open-source language models) is that it enables a simplemechanism to evolve semantically-rich text representations (with fewdomain-specific tweaks), and naturally benefits from current progress inlanguage models. Experiments in this paper highlight the versatility oflanguage-model crossover, through evolving binary bit-strings, sentences,equations, text-to-image prompts, and Python code. The conclusion is thatlanguage model crossover is a promising method for evolving genomesrepresentable as text.