Abstract
Though diffusion models have been successfully applied to various imagerestoration (IR) tasks, their performance is sensitive to the choice oftraining datasets. Typically, diffusion models trained in specific datasetsfail to recover images that have out-of-distribution degradations. To addressthis problem, this work leverages a capable vision-language model and asynthetic degradation pipeline to learn image restoration in the wild (wildIR). More specifically, all low-quality images are simulated with a syntheticdegradation pipeline that contains multiple common degradations such as blur,resize, noise, and JPEG compression. Then we introduce robust training for adegradation-aware CLIP model to extract enriched image content features toassist high-quality image restoration. Our base diffusion model is the imagerestoration SDE (IR-SDE). Built upon it, we further present a posteriorsampling strategy for fast noise-free image generation. We evaluate our modelon both synthetic and real-world degradation datasets. Moreover, experiments onthe unified image restoration task illustrate that the proposed posteriorsampling improves image generation quality for various degradations.