We present a novel neural network architecture for the image signalprocessing (ISP) pipeline. In a camera system, the ISP is a critical componentthat forms a high quality RGB image from RAW camera sensor data. Typical ISPpipelines sequentially apply a complex set of traditional image processingmodules, such as demosaicing, denoising, tone mapping, etc. We introduce a newdeep network that replaces all these modules, dubbed Deep Image Formation AndRetouching (DIFAR). DIFAR introduces a multi-scale context-aware pixel-levelblock for local denoising/demosaicing operations and a retouching block forglobal refinement of image colour, luminance and saturation. DIFAR can also betrained for RGB to RGB image enhancement. DIFAR is parameter-efficient andoutperforms recently proposed deep learning approaches in both objective andperceptual metrics, setting new state-of-the-art performance on multipledatasets including Samsung S7 and MIT-Adobe 5k.