Abstract
Hierarchical inductive biases are hypothesized to promote generalizablepolicies in reinforcement learning, as demonstrated by explicit hyperboliclatent representations and architectures. Therefore, a more flexible approachis to have these biases emerge naturally from the algorithm. We introduce FreeRandom Projection, an input mapping grounded in free probability theory thatconstructs random orthogonal matrices where hierarchical structure arisesinherently. The free random projection integrates seamlessly into existingin-context reinforcement learning frameworks by encoding hierarchicalorganization within the input space without requiring explicit architecturalmodifications. Empirical results on multi-environment benchmarks show that freerandom projection consistently outperforms the standard random projection,leading to improvements in generalization. Furthermore, analyses withinlinearly solvable Markov decision processes and investigations of the spectrumof kernel random matrices reveal the theoretical underpinnings of free randomprojection's enhanced performance, highlighting its capacity for effectiveadaptation in hierarchically structured state spaces.