Abstract
Language models continue to hallucinate despite increases in parameters,compute, and data. We propose neural diversity -- decorrelated parallelrepresentations -- as a principled mechanism that reduces hallucination ratesat fixed parameter and data budgets. Inspired by portfolio theory, whereuncorrelated assets reduce risk by $\sqrt{P}$, we prove hallucinationprobability is bounded by representational correlation: $P(H) \leqf(\sigma^2((1-\rho(P))/P + \rho(P)), \mu^2)$, which predicts that languagemodels need an optimal amount of neurodiversity. To validate this, we introduceND-LoRA (Neural Diversity Low-Rank Adaptation), combining parallel LoRAadapters with Barlow Twins regularization, and demonstrate that ND-LoRA reduceshallucinations by up to 25.6% (and 14.6% on average) without degrading generalaccuracy. Ablations show LoRA adapters and regularization act synergistically,causal interventions prove neurodiversity as the mediating factor andcorrelational analyses indicate scale: a 0.1% neural correlation increase isassociated with a 3.8% hallucination increase. Finally, task-dependentoptimality emerges: different tasks require different amounts of optimalneurodiversity. Together, our results highlight neural diversity as a thirdaxis of scaling -- orthogonal to parameters and data -- to improve thereliability of language models at fixed budgets.