Abstract
Deep neural networks exhibit exceptional accuracy when they are trained andtested on the same data distributions. However, neural classifiers are oftenextremely brittle when confronted with domain shift---changes in the inputdistribution that occur over time. We present ACE, a framework for semanticsegmentation that dynamically adapts to changing environments over the time. Byaligning the distribution of labeled training data from the original sourcedomain with the distribution of incoming data in a shifted domain, ACEsynthesizes labeled training data for environments as it sees them. Thisstylized data is then used to update a segmentation model so that it performswell in new environments. To avoid forgetting knowledge from past environments,we introduce a memory that stores feature statistics from previously seendomains. These statistics can be used to replay images in any of the previouslyobserved domains, thus preventing catastrophic forgetting. In addition tostandard batch training using stochastic gradient decent (SGD), we alsoexperiment with fast adaptation methods based on adaptive meta-learning.Extensive experiments are conducted on two datasets from SYNTHIA, the resultsdemonstrate the effectiveness of the proposed approach when adapting to anumber of tasks.