Abstract
Innate values describe agents' intrinsic motivations, which reflect theirinherent interests and preferences to pursue goals and drive them to developdiverse skills satisfying their various needs. The essence of reinforcementlearning (RL) is learning from interaction based on reward-driven behaviors,much like natural agents. It is an excellent model to describe theinnate-values-driven (IV) behaviors of AI agents. Especially developing theawareness of the AI agent through balancing internal and external utilitiesbased on its needs in different tasks is a crucial problem for individualslearning to support AI agents integrating human society with safety and harmonyin the long term. This paper proposes a hierarchical compound intrinsic valuereinforcement learning model -- innate-values-driven reinforcement learningtermed IVRL to describe the complex behaviors of AI agents' interaction. Weformulated the IVRL model and proposed two IVRL models: DQN and A2C. Bycomparing them with benchmark algorithms such as DQN, DDQN, A2C, and PPO in theRole-Playing Game (RPG) reinforcement learning test platform VIZDoom, wedemonstrated that rationally organizing various individual needs caneffectively achieve better performance.