Abstract
Model-free deep reinforcement learning has achieved great success in manydomains, such as video games, recommendation systems and robotic control tasks.In continuous control tasks, widely used policies with Gaussian distributionsresults in ineffective exploration of environments and limited performance ofalgorithms in many cases. In this paper, we propose a density-free off-policyalgorithm, Generative Actor-Critic(GAC), using the push-forward model toincrease the expressiveness of policies, which also includes an entropy-liketechnique, MMD-entropy regularizer, to balance the exploration andexploitation. Additionnally, we devise an adaptive mechanism to automaticallyscale this regularizer, which further improves the stability and robustness ofGAC. The experiment results show that push-forward policies possess desirablefeatures, such as multi-modality, which can improve the efficiency ofexploration and asymptotic performance of algorithms obviously.