Abstract

The integration of information acquired with different modalities, spatialresolution and spectral bands has shown to improve predictive accuracies. Datafusion is therefore one of the key challenges in remote sensing. Most priorwork focusing on multi-modal fusion, assumes that modalities are alwaysavailable during inference. This assumption limits the applications ofmulti-modal models since in practice the data collection process is likely togenerate data with missing, incomplete or corrupted modalities. In this paper,we show that Generative Adversarial Networks can be effectively used toovercome the problems that arise when modalities are missing or incomplete.Focusing on semantic segmentation of building footprints with missingmodalities, our approach achieves an improvement of about 2% on theIntersection over Union (IoU) against the same network that relies only on theavailable modality.

 

Introduction (beta)

None

 

Conclusion (beta)

None