Abstract
The use of semantic segmentation for masking and cropping input images hasproven to be a significant aid in medical imaging classification tasks bydecreasing the noise and variance of the training dataset. However,implementing this approach with classical methods is challenging: the cost ofobtaining a dense segmentation is high, and the precise input area that is mostcrucial to the classification task is difficult to determine a-priori. Wepropose a novel joint-training deep reinforcement learning framework for imageaugmentation. A segmentation network, weakly supervised with policy gradientoptimization, acts as an agent, and outputs masks as actions given samples asstates, with the goal of maximizing reward signals from the classificationnetwork. In this way, the segmentation network learns to mask unimportantimaging features. Our method, Adversarial Policy Gradient Augmentation (APGA),shows promising results on Stanford's MURA dataset and on a hip fractureclassification task with an increase in global accuracy of up to 7.33% andimproved performance over baseline methods in 9/10 tasks evaluated. We discussthe broad applicability of our joint training strategy to a variety of medicalimaging tasks.