Abstract
Personalized probabilistic forecasts of time to event (such as mortality) canbe crucial in decision making, especially in the clinical setting. Inspired byideas from the meteorology literature, we approach this problem through theparadigm of maximizing sharpness of prediction distributions, subject tocalibration. In regression problems, it has been shown that optimizing thecontinuous ranked probability score (CRPS) instead of maximum likelihood leadsto sharper prediction distributions while maintaining calibration. We introducethe Survival-CRPS, a generalization of the CRPS to the time to event setting,and present right-censored and interval-censored variants. To holisticallyevaluate the quality of predicted distributions over time to event, we presentthe Survival-AUPRC evaluation metric, an analog to area under theprecision-recall curve. We apply these ideas by building a recurrent neuralnetwork for mortality prediction, using an Electronic Health Record datasetcovering millions of patients. We demonstrate significant benefits in modelstrained by the Survival-CRPS objective instead of maximum likelihood.