Abstract
Foundation models, such as large language models, have demonstrated successin addressing various language and image processing tasks. In this work, weintroduce a multi-modal foundation model for scientific problems, namedPROSE-PDE. Our model, designed for bi-modality to bi-modality learning, is amulti-operator learning approach which can predict future states ofspatiotemporal systems while concurrently learning the underlying governingequations of the physical system. Specifically, we focus on multi-operatorlearning by training distinct one-dimensional time-dependent nonlinear constantcoefficient partial differential equations, with potential applications to manyphysical applications including physics, geology, and biology. Moreimportantly, we provide three extrapolation studies to demonstrate thatPROSE-PDE can generalize physical features through the robust training ofmultiple operators and that the proposed model can extrapolate to predict PDEsolutions whose models or data were unseen during the training. Furthermore, weshow through systematic numerical experiments that the utilization of thesymbolic modality in our model effectively resolves the well-posedness problemswith training multiple operators and thus enhances our model's predictivecapabilities.