The reinforcement learning research area contains a wide range of methods forsolving the problems of intelligent agent control. Despite the progress thathas been made, the task of creating a highly autonomous agent is still asignificant challenge. One potential solution to this problem is intrinsicmotivation, a concept derived from developmental psychology. This reviewconsiders the existing methods for determining intrinsic motivation based onthe world model obtained by the agent. We propose a systematic approach tocurrent research in this field, which consists of three categories of methods,distinguished by the way they utilize a world model in the agent's components:complementary intrinsic reward, exploration policy, and intrinsically motivatedgoals. The proposed unified framework describes the architecture of agentsusing a world model and intrinsic motivation to improve learning. The potentialfor developing new techniques in this area of research is also examined.