Abstract
We developed a simulated environment to train deep reinforcement learningagents on a shortcut usage navigation task, motivated by the Dual SolutionsParadigm test used for human navigators. We manipulated the frequency withwhich agents were exposed to a shortcut and a navigation cue, to investigatehow these factors influence shortcut usage development. We find that all agentsrapidly achieve optimal performance in closed shortcut trials once initiallearning starts. However, their navigation speed and shortcut usage when it isopen happen faster in agents with higher shortcut exposure. Analysis of theagents' artificial neural networks activity revealed that frequent presentationof a cue initially resulted in better encoding of the cue in the activity ofindividual nodes, compared to agents who encountered the cue less often.However, stronger cue representations were ultimately formed through the use ofthe cue in the context of navigation planning, rather than simply throughexposure. We found that in all agents, spatial representations develop early intraining and subsequently stabilize before navigation strategies fully develop,suggesting that having spatially consistent activations is necessary for basicnavigation, but insufficient for advanced strategies. Further, using newanalysis techniques, we found that the planned trajectory rather than theagent's immediate location is encoded in the agent's networks. Moreover, theencoding is represented at the population rather than the individual nodelevel. These techniques could have broader applications in studying neuralactivity across populations of neurons or network nodes beyond individualactivity patterns.