Abstract
Domain adaptation for semantic segmentation across datasets consisting of thesame categories has seen several recent successes. However, a more generalscenario is when the source and target datasets correspond to non-overlappinglabel spaces. For example, categories in segmentation datasets change vastlydepending on the type of environment or application, yet share many valuablesemantic relations. Existing approaches based on feature alignment ordiscrepancy minimization do not take such category shift into account. In thiswork, we present Cluster-to-Adapt (C2A), a computationally efficientclustering-based approach for domain adaptation across segmentation datasetswith completely different, but possibly related categories. We show that such aclustering objective enforced in a transformed feature space serves toautomatically select categories across source and target domains that can bealigned for improving the target performance, while preventing negativetransfer for unrelated categories. We demonstrate the effectiveness of ourapproach through experiments on the challenging problem of outdoor to indooradaptation for semantic segmentation in few-shot as well as zero-shot settings,with consistent improvements in performance over existing approaches andbaselines in all cases.