Abstract
Background: In medical imaging, prior studies have demonstrated disparate AIperformance by race, yet there is no known correlation for race on medicalimaging that would be obvious to the human expert interpreting the images. Methods: Using private and public datasets we evaluate: A) performancequantification of deep learning models to detect race from medical images,including the ability of these models to generalize to external environmentsand across multiple imaging modalities, B) assessment of possible confoundinganatomic and phenotype population features, such as disease distribution andbody habitus as predictors of race, and C) investigation into the underlyingmechanism by which AI models can recognize race. Findings: Standard deep learning models can be trained to predict race frommedical images with high performance across multiple imaging modalities. Ourfindings hold under external validation conditions, as well as when models areoptimized to perform clinically motivated tasks. We demonstrate this detectionis not due to trivial proxies or imaging-related surrogate covariates for race,such as underlying disease distribution. Finally, we show that performancepersists over all anatomical regions and frequency spectrum of the imagessuggesting that mitigation efforts will be challenging and demand furtherstudy. Interpretation: We emphasize that model ability to predict self-reported raceis itself not the issue of importance. However, our findings that AI cantrivially predict self-reported race -- even from corrupted, cropped, andnoised medical images -- in a setting where clinical experts cannot, creates anenormous risk for all model deployments in medical imaging: if an AI modelsecretly used its knowledge of self-reported race to misclassify all Blackpatients, radiologists would not be able to tell using the same data the modelhas access to.