Abstract
Current deep learning research is dominated by benchmark evaluation. A methodis regarded as favorable if it empirically performs well on the dedicated testset. This mentality is seamlessly reflected in the resurfacing area ofcontinual learning, where consecutively arriving sets of benchmark data areinvestigated. The core challenge is framed as protecting previously acquiredrepresentations from being catastrophically forgotten due to the iterativeparameter updates. However, comparison of individual methods is neverthelesstreated in isolation from real world application and typically judged bymonitoring accumulated test set performance. The closed world assumptionremains predominant. It is assumed that during deployment a model is guaranteedto encounter data that stems from the same distribution as used for training.This poses a massive challenge as neural networks are well known to provideoverconfident false predictions on unknown instances and break down in the faceof corrupted data. In this work we argue that notable lessons from open setrecognition, the identification of statistically deviating data outside of theobserved dataset, and the adjacent field of active learning, where data isincrementally queried such that the expected performance gain is maximized, arefrequently overlooked in the deep learning era. Based on these forgottenlessons, we propose a consolidated view to bridge continual learning, activelearning and open set recognition in deep neural networks. Our results showthat this not only benefits each individual paradigm, but highlights thenatural synergies in a common framework. We empirically demonstrateimprovements when alleviating catastrophic forgetting, querying data in activelearning, selecting task orders, while exhibiting robust open world applicationwhere previously proposed methods fail.