Abstract
Deep hierarchical reinforcement learning has gained a lot of attention inrecent years due to its ability to produce state-of-the-art results inchallenging environments where non-hierarchical frameworks fail to learn usefulpolicies. However, as problem domains become more complex, deep hierarchicalreinforcement learning can become inefficient, leading to longer convergencetimes and poor performance. We introduce the Deep Nested Agent framework, whichis a variant of deep hierarchical reinforcement learning where information fromthe main agent is propagated to the low level $nested$ agent by incorporatingthis information into the nested agent's state. We demonstrate theeffectiveness and performance of the Deep Nested Agent framework by applying itto three scenarios in Minecraft with comparisons to a deep non-hierarchicalsingle agent framework, as well as, a deep hierarchical framework.