Sparsely Changing Latent States for Prediction and Planning in Partially Observable Domains

  • 2022-01-13 17:36:29
  • Christian Gumbsch, Martin V. Butz, Georg Martius
  • 0

Abstract

A common approach to prediction and planning in partially observable domainsis to use recurrent neural networks (RNNs), which ideally develop and maintaina latent memory about hidden, task-relevant factors. We hypothesize that manyof these hidden factors in the physical world are constant over time, changingonly sparsely. To study this hypothesis, we propose Gated $L_0$ RegularizedDynamics (GateL0RD), a novel recurrent architecture that incorporates theinductive bias to maintain stable, sparsely changing latent states. The bias isimplemented by means of a novel internal gating function and a penalty on the$L_0$ norm of latent state changes. We demonstrate that GateL0RD can competewith or outperform state-of-the-art RNNs in a variety of partially observableprediction and control tasks. GateL0RD tends to encode the underlyinggenerative factors of the environment, ignores spurious temporal dependencies,and generalizes better, improving sampling efficiency and overall performancein model-based planning and reinforcement learning tasks. Moreover, we showthat the developing latent states can be easily interpreted, which is a steptowards better explainability in RNNs.

 

Quick Read (beta)

loading the full paper ...