Abstract
Domain adaptation is an important task to enable learning when labels arescarce. While most works focus only on the image modality, there are manyimportant multi-modal datasets. In order to leverage multi-modality for domainadaptation, we propose cross-modal learning, where we enforce consistencybetween the predictions of two modalities via mutual mimicking. We constrainour network to make correct predictions on labeled data and consistentpredictions across modalities on unlabeled target-domain data. Experiments inunsupervised and semi-supervised domain adaptation settings prove theeffectiveness of this novel domain adaptation strategy. Specifically, weevaluate on the task of 3D semantic segmentation using the image and pointcloud modality. We leverage recent autonomous driving datasets to produce awide variety of domain adaptation scenarios including changes in scene layout,lighting, sensor setup and weather, as well as the synthetic-to-real setup. Ourmethod significantly improves over previous uni-modal adaptation baselines onall adaption scenarios. Code will be made available.