Abstract
Transformers have demonstrated remarkable performance in natural languageprocessing and related domains, as they largely focus on sequential,autoregressive next-token prediction tasks. Yet, they struggle in logicalreasoning, not necessarily because of a fundamental limitation of these models,but possibly due to the lack of exploration of more creative uses, such aslatent space and recurrent reasoning. An emerging exploration in this directionis the Hierarchical Reasoning Model (Wang et. al., 2025), which introduces anovel type of recurrent reasoning in the latent space of transformers,achieving remarkable performance on a wide range of 2D reasoning tasks. Despitethe promising results, this line of models is still at an early stage and callsfor in-depth investigation. In this work, we review this class of models,examine key design choices, test alternative variants and clarify commonmisconceptions.