Abstract
Analogical reasoning-the capacity to identify and map structuralrelationships between different domains-is fundamental to human cognition andlearning. Recent studies have shown that large language models (LLMs) cansometimes match humans in analogical reasoning tasks, opening the possibilitythat analogical reasoning might emerge from domain general processes. However,it is still debated whether these emergent capacities are largely superficialand limited to simple relations seen during training or whether they ratherencompass the flexible representational and mapping capabilities which are thefocus of leading cognitive models of analogy. In this study, we introduce novelanalogical reasoning tasks that require participants to map betweensemantically contentful words and sequences of letters and other abstractcharacters. This task necessitates the ability to flexibly re-represent richsemantic information-an ability which is known to be central to human analogybut which is thus far not well-captured by existing cognitive theories andmodels. We assess the performance of both human participants and LLMs on tasksfocusing on reasoning from semantic structure and semantic content, introducingvariations that test the robustness of their analogical inferences. AdvancedLLMs match human performance across several conditions, though humans and LLMsrespond differently to certain task variations and semantic distractors. Ourresults thus provide new evidence that LLMs might offer a how-possiblyexplanation of human analogical reasoning in contexts that are not yet wellmodeled by existing theories, but that even today's best models are unlikely toyield how-actually explanations.