Memory-Based Meta-Learning on Non-Stationary Distributions

  • 2023-05-25 18:53:32
  • Tim Genewein, Gr├ęgoire Del├ętang, Anian Ruoss, Li Kevin Wenliang, Elliot Catt, Vincent Dutordoir, Jordi Grau-Moya, Laurent Orseau, Marcus Hutter, Joel Veness
  • 0

Abstract

Memory-based meta-learning is a technique for approximating Bayes-optimalpredictors. Under fairly general conditions, minimizing sequential predictionerror, measured by the log loss, leads to implicit meta-learning. The goal ofthis work is to investigate how far this interpretation can be realized bycurrent sequence prediction models and training regimes. The focus is onpiecewise stationary sources with unobserved switching-points, which arguablycapture an important characteristic of natural language and action-observationsequences in partially observable environments. We show that various types ofmemory-based neural models, including Transformers, LSTMs, and RNNs can learnto accurately approximate known Bayes-optimal algorithms and behave as ifperforming Bayesian inference over the latent switching-points and the latentparameters governing the data distribution within each segment.

 

Quick Read (beta)

loading the full paper ...