Combining Noise-to-Image and Image-to-Image GANs: Brain MR Image Augmentation for Tumor Detection

  • 2019-09-19 17:53:49
  • Changhee Han, Leonardo Rundo, Ryosuke Araki, Yudai Nagano, Yujiro Furukawa, Giancarlo Mauri, Hideki Nakayama, Hideaki Hayashi
  • 0

Abstract

Convolutional Neural Networks (CNNs) achieve excellent computer-assisteddiagnosis with sufficient annotated training data. However, most medicalimaging datasets are small and fragmented. In this context, GenerativeAdversarial Networks (GANs) can synthesize realistic/diverse additionaltraining images to fill the data lack in the real image distribution;researchers have improved classification by augmenting data with noise-to-image(e.g., random noise samples to diverse pathological images) or image-to-imageGANs (e.g., a benign image to a malignant one). Yet, no research has reportedresults combining noise-to-image and image-to-image GANs for furtherperformance boost. Therefore, to maximize the DA effect with the GANcombinations, we propose a two-step GAN-based DA that generates and refinesbrain Magnetic Resonance (MR) images with/without tumors separately: (i)Progressive Growing of GANs (PGGANs), multi-stage noise-to-image GAN forhigh-resolution MR image generation, first generates realistic/diverse 256 X256 images; (ii) Multimodal UNsupervised Image-to-image Translation (MUNIT)that combines GANs/Variational AutoEncoders or SimGAN that uses a DA-focusedGAN loss, further refines the texture/shape of the PGGAN-generated imagessimilarly to the real ones. We thoroughly investigate CNN-based tumorclassification results, also considering the influence of pre-training onImageNet and discarding weird-looking GAN-generated images. The results showthat, when combined with classic DA, our two-step GAN-based DA cansignificantly outperform the classic DA alone, in tumor detection (i.e.,boosting sensitivity 93.67% to 97.48%) and also in other medical imaging tasks.

 

Quick Read (beta)

loading the full paper ...