Abstract
Advances in medical imaging technologies have enabled the collection oflongitudinal images, which involve repeated scanning of the same patients overtime, to monitor disease progression. However, predictive modeling of such dataremains challenging due to high dimensionality, irregular sampling, and datasparsity. To address these issues, we propose ImageFlowNet, a novel modeldesigned to forecast disease trajectories from initial images while preservingspatial details. ImageFlowNet first learns multiscale joint representationspaces across patients and time points, then optimizes deterministic orstochastic flow fields within these spaces using a position-parameterizedneural ODE/SDE framework. The model leverages a UNet architecture to createrobust multiscale representations and mitigates data scarcity by combiningknowledge from all patients. We provide theoretical insights that support ourformulation of ODEs, and motivate our regularizations involving high-levelvisual features, latent space organization, and trajectory smoothness. Wevalidate ImageFlowNet on three longitudinal medical image datasets depictingprogression in geographic atrophy, multiple sclerosis, and glioblastoma,demonstrating its ability to effectively forecast disease progression andoutperform existing methods. Our contributions include the development ofImageFlowNet, its theoretical underpinnings, and empirical validation onreal-world datasets. The official implementation is available athttps://github.com/KrishnaswamyLab/ImageFlowNet.