Abstract
Imaging-based deep learning has transformed healthcare research, yet itsclinical adoption remains limited due to challenges in comparing imaging modelswith traditional non-imaging and tabular data. To bridge this gap, we introduceBarttender, an interpretable framework that uses deep learning for the directcomparison of the utility of imaging versus non-imaging tabular data for taskslike disease prediction. Barttender converts non-imaging tabular features, such as scalar data fromelectronic health records, into grayscale bars, facilitating an interpretableand scalable deep learning based modeling of both data modalities. Ourframework allows researchers to evaluate differences in utility throughperformance measures, as well as local (sample-level) and global(population-level) explanations. We introduce a novel measure to define globalfeature importances for image-based deep learning models, which we call gIoU.Experiments on the CheXpert and MIMIC datasets with chest X-rays and scalardata from electronic health records show that Barttender performs comparably totraditional methods and offers enhanced explainability using deep learningmodels.