Abstract
One problem in the application of reinforcement learning to real-worldproblems is the curse of dimensionality on the action space. Macro actions, asequence of primitive actions, have been studied to diminish the dimensionalityof the action space with regard to the time axis. However, previous studiesrelied on humans defining macro actions or assumed macro actions as repetitionsof the same primitive actions. We present Factorized Macro Action ReinforcementLearning (FaMARL) which autonomously learns disentangled factor representationof a sequence of actions to generate macro actions that can be directly appliedto general reinforcement learning algorithms. FaMARL exhibits higher scoresthan other reinforcement learning algorithms on environments that require anextensive amount of search.