Abstract
With the extremely rapid advances in remote sensing (RS) technology, a greatquantity of Earth observation (EO) data featuring considerable and complicatedheterogeneity is readily available nowadays, which renders researchers anopportunity to tackle current geoscience applications in a fresh way. With thejoint utilization of EO data, much research on multimodal RS data fusion hasmade tremendous progress in recent years, yet these developed traditionalalgorithms inevitably meet the performance bottleneck due to the lack of theability to comprehensively analyse and interpret these strongly heterogeneousdata. Hence, this non-negligible limitation further arouses an intense demandfor an alternative tool with powerful processing competence. Deep learning(DL), as a cutting-edge technology, has witnessed remarkable breakthroughs innumerous computer vision tasks owing to its impressive ability in datarepresentation and reconstruction. Naturally, it has been successfully appliedto the field of multimodal RS data fusion, yielding great improvement comparedwith traditional methods. This survey aims to present a systematic overview inDL-based multimodal RS data fusion. More specifically, some essential knowledgeabout this topic is first given. Subsequently, a literature survey is conductedto analyse the trends of this field. Some prevalent sub-fields in themultimodal RS data fusion are then reviewed in terms of the to-be-fused datamodalities, i.e., spatiospectral, spatiotemporal, light detection andranging-optical, synthetic aperture radar-optical, and RS-Geospatial Big Datafusion. Furthermore, We collect and summarize some valuable resources for thesake of the development in multimodal RS data fusion. Finally, the remainingchallenges and potential future directions are highlighted.