Abstract
We aim to generate high resolution shallow depth-of-field (DoF) images from asingle all-in-focus image with controllable focal distance and aperture size.To achieve this, we propose a novel neural network model comprised of a depthprediction module, a lens blur module, and a guided upsampling module. Allmodules are differentiable and are learned from data. To train our depthprediction module, we collect a dataset of 2462 RGB-D images captured by mobilephones with a dual-lens camera, and use existing segmentation datasets toimprove border prediction. We further leverage a synthetic dataset with knowndepth to supervise the lens blur and guided upsampling modules. Theeffectiveness of our system and training strategies are verified in theexperiments. Our method can generate high-quality shallow DoF images at highresolution, and produces significantly fewer artifacts than the baselines andexisting solutions for single image shallow DoF synthesis. Compared with theiPhone portrait mode, which is a state-of-the-art shallow DoF solution based ona dual-lens depth camera, our method generates comparable results, whileallowing for greater flexibility to choose focal points and aperture size, andis not limited to one capture setup.