Abstract
Continual learning experiments used in current deep learning papers do notfaithfully assess fundamental challenges of learning continually, maskingweak-points of the suggested approaches instead. We study gaps in such existingevaluations, proposing essential experimental evaluations that are morerepresentative of continual learning's challenges, and suggest are-prioritization of research efforts in the field. We show that currentapproaches fail with our new evaluations and, to analyse these failures, wepropose a variational loss which unifies many existing solutions to continuallearning under a Bayesian framing, as either 'prior-focused' or'likelihood-focused'. We show that while prior-focused approaches such as EWCand VCL perform well on existing evaluations, they perform dramatically worsewhen compared to likelihood-focused approaches on other simple tasks.