Abstract
Image deraining is a new challenging problem in applications of autonomousvehicles. In a bad weather condition of heavy rainfall, raindrops, mainlyhitting the vehicle's windshield, can significantly reduce observation abilityeven though the windshield wipers might be able to remove part of it. Moreover,rain flows spreading over the windshield can yield the physical effect ofrefraction, which seriously impede the sightline or undermine the machinelearning system equipped in the vehicle. In this paper, we propose a newmulti-stage multi-task recurrent generative adversarial network (M2GAN) to dealwith challenging problems of raindrops hitting the car's windshield. Thismethod is also applicable for removing raindrops appearing on a glass window orlens. M2GAN is a multi-stage multi-task generative adversarial network that canutilize prior high-level information, such as semantic segmentation, to boostderaining performance. To demonstrate M2GAN, we introduce the first real-worlddataset for rain removal on autonomous vehicles. The experimental results showthat our proposed method is superior to other state-of-the-art approaches ofderaining raindrops in respect of quantitative metrics and visual quality.M2GAN is considered the first method to deal with challenging problems ofreal-world rains under unconstrained environments such as autonomous vehicles.