Abstract
Deep learning has enabled breakthroughs in automated diagnosis from medicalimaging, with many successful applications in ophthalmology. However, standardmedical image classification approaches only assess disease presence at thetime of acquisition, neglecting the common clinical setting of longitudinalimaging. For slow, progressive eye diseases like age-related maculardegeneration (AMD) and primary open-angle glaucoma (POAG), patients undergorepeated imaging over time to track disease progression and forecasting thefuture risk of developing disease is critical to properly plan treatment. Ourproposed Longitudinal Transformer for Survival Analysis (LTSA) enables dynamicdisease prognosis from longitudinal medical imaging, modeling the time todisease from sequences of fundus photography images captured over long,irregular time periods. Using longitudinal imaging data from the Age-RelatedEye Disease Study (AREDS) and Ocular Hypertension Treatment Study (OHTS), LTSAsignificantly outperformed a single-image baseline in 19/20 head-to-headcomparisons on late AMD prognosis and 18/20 comparisons on POAG prognosis. Atemporal attention analysis also suggested that, while the most recent image istypically the most influential, prior imaging still provides additionalprognostic value.