Abstract
As machine learning models in critical fields increasingly grapple withmultimodal data, they face the dual challenges of handling a wide array ofmodalities, often incomplete due to missing elements, and the temporalirregularity and sparsity of collected samples. Successfully leveraging thiscomplex data, while overcoming the scarcity of high-quality training samples,is key to improving these models' predictive performance. We introduce``FuseMoE'', a mixture-of-experts framework incorporated with an innovativegating function. Designed to integrate a diverse number of modalities, FuseMoEis effective in managing scenarios with missing modalities and irregularlysampled data trajectories. Theoretically, our unique gating functioncontributes to enhanced convergence rates, leading to better performance inmultiple downstream tasks. The practical utility of FuseMoE in the real worldis validated by a diverse set of challenging prediction tasks.