Abstract
When faced with novel situations, people are able to marshal relevantconsiderations from a wide range of background knowledge and put these to usein inferences and predictions. What permits us to draw in globally relevantinformation and reason over it coherently? Here, we explore the hypothesis thatpeople use a combination of distributed and symbolic representations toconstruct bespoke mental models tailored to novel situations. We propose acomputational implementation of this idea -- a ``Model Synthesis Architecture''(MSA) -- using language models to implement global relevance-based retrievaland model synthesis and probabilistic programs to implement bespoke, coherentworld models. We evaluate our MSA as a model of human judgments on a novelreasoning dataset. The dataset -- built around a `Model Olympics` domain ofsports vignettes -- tests models' capacity for human-like, open-ended reasoningby requiring (i) judgments about novel causal structures described in language;(ii) drawing on large bodies of background knowledge; and (iii) doing both inlight of observations that introduce arbitrary novel variables. Our MSAapproach captures human judgments better than language model-only baselines,under both direct and chain-of-thought generations from the LM that supportsmodel synthesis. These results suggest that MSAs can be implemented in a waythat mirrors people's ability to deliver locally coherent reasoning overglobally relevant variables, offering a path to understanding and replicatinghuman reasoning in open-ended domains.