Estimating Risk and Uncertainty in Deep Reinforcement Learning

  • 2020-09-09 15:44:27
  • William R. Clements, Bastien Van Delft, Benoît-Marie Robaglia, Reda Bahi Slaoui, Sébastien Toth
  • 0

Abstract

Reinforcement learning agents are faced with two types of uncertainty.Epistemic uncertainty stems from limited data and is useful for exploration,whereas aleatoric uncertainty arises from stochastic environments and must beaccounted for in risk-sensitive applications. We highlight the challengesinvolved in simultaneously estimating both of them, and propose a framework fordisentangling and estimating these uncertainties on learned Q-values. We deriveunbiased estimators of these uncertainties and introduce an uncertainty-awareDQN algorithm, which we show exhibits safe learning behavior and outperformsother DQN variants on the MinAtar testbed.

 

Quick Read (beta)

loading the full paper ...