Abstract
Automated radiology report generation from chest X-ray (CXR) images has thepotential to improve clinical efficiency and reduce radiologists' workload.However, most datasets, including the publicly available MIMIC-CXR and CheXpertPlus, consist entirely of free-form reports, which are inherently variable andunstructured. This variability poses challenges for both generation andevaluation: existing models struggle to produce consistent, clinicallymeaningful reports, and standard evaluation metrics fail to capture the nuancesof radiological interpretation. To address this, we introduce StructuredRadiology Report Generation (SRRG), a new task that reformulates free-textradiology reports into a standardized format, ensuring clarity, consistency,and structured clinical reporting. We create a novel dataset by restructuringreports using large language models (LLMs) following strict structuredreporting desiderata. Additionally, we introduce SRR-BERT, a fine-graineddisease classification model trained on 55 labels, enabling more precise andclinically informed evaluation of structured reports. To assess report quality,we propose F1-SRR-BERT, a metric that leverages SRR-BERT's hierarchical diseasetaxonomy to bridge the gap between free-text variability and structuredclinical reporting. We validate our dataset through a reader study conducted byfive board-certified radiologists and extensive benchmarking experiments.