Abstract
A common belief in model-free reinforcement learning is that methods based onrandom search in the parameter space of policies exhibit significantly worsesample complexity than those that explore the space of actions. We dispel suchbeliefs by introducing a random search method for training static, linearpolicies for continuous control problems, matching state-of-the-art sampleefficiency on the benchmark MuJoCo locomotion tasks. Our method also finds anearly optimal controller for a challenging instance of the Linear QuadraticRegulator, a classical problem in control theory, when the dynamics are notknown. Computationally, our random search algorithm is at least 15 times moreefficient than the fastest competing model-free methods on these benchmarks. Wetake advantage of this computational efficiency to evaluate the performance ofour method over hundreds of random seeds and many different hyperparameterconfigurations for each benchmark task. Our simulations highlight a highvariability in performance in these benchmark tasks, suggesting that commonlyused estimations of sample efficiency do not adequately evaluate theperformance of RL algorithms.