Neural Blind Deconvolution Using Deep Priors

  • 2019-08-06 15:03:44
  • Dongwei Ren, Kai Zhang, Qilong Wang, Qinghua Hu, Wangmeng Zuo
  • 64

Abstract

Blind deconvolution is a classical yet challenging low-level vision problemwith many real-world applications. Traditional maximum a posterior (MAP) basedmethods rely heavily on fixed and handcrafted priors that certainly areinsufficient in characterizing clean images and blur kernels, and usually adoptspecially designed alternating minimization to avoid trivial solution. Incontrast, existing deep motion deblurring networks learn from massive trainingimages the mapping to clean image or blur kernel, but are limited in handlingvarious complex and large size blur kernels. To connect MAP and deep models, wein this paper present two generative networks for respectively modeling thedeep priors of clean image and blur kernel, and propose an unconstrained neuraloptimization solution to blind deconvolution. In particular, we adopt anasymmetric Autoencoder with skip connections for generating latent clean image,and a fully-connected network (FCN) for generating blur kernel. Moreover, theSoftMax nonlinearity is applied to the output layer of FCN to meet thenon-negative and equality constraints. The process of neural optimization canbe explained as a kind of "zero-shot" self-supervised learning of thegenerative networks, and thus our proposed method is dubbed SelfDeblur.Experimental results show that our SelfDeblur can achieve notable quantitativegains as well as more visually plausible deblurring results in comparison tostate-of-the-art blind deconvolution methods on benchmark datasets andreal-world blurry images. The source code is available athttps://github.com/csdwren/SelfDeblur.

 

Quick Read (beta)

Neural Blind Deconvolution Using Deep Priors

Dongwei Ren College of Intelligence and Computing, Tianjin University, Tianjin, China Kai Zhang School of Computer Science and Technology, Harbin Institute of Technology, Harbin, China Qilong Wang College of Intelligence and Computing, Tianjin University, Tianjin, China Qinghua Hu College of Intelligence and Computing, Tianjin University, Tianjin, China Wangmeng Zuo School of Computer Science and Technology, Harbin Institute of Technology, Harbin, China
Abstract

Blind deconvolution is a classical yet challenging low-level vision problem with many real-world applications. Traditional maximum a posterior (MAP) based methods rely heavily on fixed and handcrafted priors that certainly are insufficient in characterizing clean images and blur kernels, and usually adopt specially designed alternating minimization to avoid trivial solution. In contrast, existing deep motion deblurring networks learn from massive training images the mapping to clean image or blur kernel, but are limited in handling various complex and large size blur kernels. To connect MAP and deep models, we in this paper present two generative networks for respectively modeling the deep priors of clean image and blur kernel, and propose an unconstrained neural optimization solution to blind deconvolution. In particular, we adopt an asymmetric Autoencoder with skip connections for generating latent clean image, and a fully-connected network (FCN) for generating blur kernel. Moreover, the SoftMax nonlinearity is applied to the output layer of FCN to meet the non-negative and equality constraints. The process of neural optimization can be explained as a kind of ”zero-shot” self-supervised learning of the generative networks, and thus our proposed method is dubbed SelfDeblur. Experimental results show that our SelfDeblur can achieve notable quantitative gains as well as more visually plausible deblurring results in comparison to state-of-the-art blind deconvolution methods on benchmark datasets and real-world blurry images. The source code is available at https://github.com/csdwren/SelfDeblur.

1 Introduction

Blurry image Xu & Jia [48]
Pan-L0[28] Sun et al. [41]
Pan-DCP[30] SelfDeblur
Figure 1: Visual quality comparison on a severe blurry image. SelfDeblur succeeds in estimating large size (75×75) blur kernel, and generates visually satisfying latent clean image.

Camera shake during exposure inevitably yields blurry images and is a long-standing annoying issue in digital photography. The removal of distortion from a blurry image, i.e., image deblurring, is a classical ill-posed problem in low-level vision and has received considerable research attention [2, 4, 20, 29, 56, 33, 30, 3, 11]. When the blur kernel is spatially invariant, it is also known as blind deconvolution, where the blurry image 𝐲 can be formulated as,

𝐲=𝐤𝐱+𝐧, (1)

where denotes the 2D convolution operator, 𝐱 is the latent clean image, 𝐤 is the blur kernel, and 𝐧 is the additive white Gaussian noise (AWGN) with noise level σ. It can be seen that blind deconvolution should estimate both 𝐤 and 𝐱 from a blurry image 𝐲, making it remain a very challenging problem after decades of studies.

Most traditional blind deconvolution methods are based on the Maximum a Posterior (MAP) framework,

(𝐤,𝐱) =argmax𝐱,𝐤Pr(𝐤,𝐱|𝐲), (2)
=argmax𝐱,𝐤Pr(𝐲|𝐤,𝐱)Pr(𝐱)Pr(𝐤),

where Pr(𝐲|𝐤,𝐱) is the likelihood corresponding to the fidelity term, and Pr(𝐱) and Pr(𝐤) model the priors of clean image and blur kernel, respectively. Although many priors have been suggested for 𝐱 [2, 28, 56, 16] and 𝐤 [56, 30, 20, 23, 41, 25, 34, 30, 50], they generally are handcrafted and certainly are insufficient in characterizing clean images and blur kernels. Furthermore, the non-convexity of MAP based models also increases the difficulty of optimization. Levin et al. [20] reveal that MAP-based methods may converge to trivial solution of delta kernel. Perrone and Favaro [33] show that the success of existing methods can be attributed to some optimization details, e.g., projected alternating minimization and delayed normalization of 𝐤.

Figure 2: Illustration of our SelfDeblur method. The generative networks 𝒢x and 𝒢k are deployed to respectively capture the deep priors of blur kernel and latent clean image, and are trained using only the input blurry image. In particular, 𝒢x is an asymmetric Autoencoder with skip connections, and the Sigmoid nonlinearity is adopted to the output layer for constraining the pixels in 𝐱 in the range [0,1]. 𝒢k adopts a fully-connected network structure, where the SoftMax nonlinearity is applied to the output layer for meeting the non-negative and equality constraints, and the 1D output of is reshaped to 2D blur kernel.

Motivated by the unprecedented success of deep learning in low-level vision [12, 52, 13, 42, 24], some attempts have also been made to solve blind deconvolution using deep convolutional networks (CNNs). Given the training set, deep CNNs can either be used to extract features to facilitate blur kernel estimation [38, 1], or be deployed to learn the direct mapping to clean image for motion deblurring [51, 26, 43, 7]. However, these methods do not succeed in handling various complex and large size blur kernels in blind deconvolution. Recently, Ulyanov et al[45] suggest the deep image prior (DIP) framework, which adopts the structure of a DIP generator network to capture low-level image statistics and shows powerful ability in image denoising, super-resolution, inpainting, etc. Subsequently, Gandelsman et al[6] combine multiple DIPs (i.e., Double-DIP) for multi-task layer decomposition such as image dehazing and transparency separation. However, Double-DIP cannot be directly applied to solve blind deconvolution due to that the DIP network is designed to generate natural images and is limited to capture the prior of blur kernels.

In this paper, we propose a novel neural optimization solution to blind deconvolution. Motivated by the DIP network [45], an image generator network 𝒢x, i.e., an asymmetric Autoencoder with skip connections, is deployed to capture the statistics of latent clean image. Nonetheless, image generator network cannot well characterize the prior on blur kernel. Instead, we adopt a fully-connected network (FCN) 𝒢k to model the prior of blur kernel. Furthermore, the Softmax nonlinearity is deployed to the output layer of 𝒢k, and the non-negative and equality constraints on blur kernel can then be naturally satisfied. By fixing the network structures (𝒢k and 𝒢x) and inputs (𝐳k and 𝐳x) sampled from uniform distribution, blind deconvolution is thus formulated as an unconstrained neural optimization on network parameters of 𝒢k and 𝒢x. As illustrated in Fig. 2, given a blurry image 𝐲, the optimization process can also be explained as a kind of ”zero-shot” self-supervised learning [39] of 𝒢k and 𝒢x, and our proposed method is dubbed SelfDeblur. Even though SelfDeblur can be optimized with either alternating optimization or joint optimization, our empirical study shows that the latter performs better in most cases.

Experiments are conducted on two widely used benchmarks [20, 19] as well as real-world blurry images to evaluate our SelfDeblur. Fig. 1 shows the deblurring results on a severe real-world blurry image. While the competing methods either fail to estimate large size blur kernels or suffer from ringing effects, our SelfDeblur succeed in estimating the blur kernel and generating visually favorable deblurring image. In comparison to the state-of-the-art methods, our SelfDeblur can achieve notable quantitative performance gains and performs favorably in generating visually plausible deblurring results. It is worth noting that our SelfDeblur can both estimate blur kernel and generate latent clean image with satisfying visual quality, making the subsequent non-blind deconvolution not a compulsory choice.

Our contributions are summarized as follows:

  • A neural blind deconvolution method, i.e., SelfDeblur, is proposed, where DIP and FCN are respectively introduced to capture the priors of clean image and blur kernel. And the SoftMax nonlinearity is applied to the output layer of FCN to meet the non-negative and equality constraints.

  • The joint optimization algorithm is suggested to solve the unconstrained neural blind deconvolution model for both estimating blur kernel and generating latent clean image, making the non-blind deconvolution not a compulsory choice for our SelfDeblur.

  • Extensive experiments show that our SelfDeblur performs favorably against the existing MAP-based methods in terms of quantitative and qualitative evaluation. To our best knowledge, SelfDeblur makes the first attempt of applying deep learning to yield state-of-the-art blind deconvolution performance.

2 Related Work

In this section, we briefly survey the relevant works including optimization-based blind deconvolution and deep learning based blind deblurring methods.

2.1 Optimization-based Blind Deconvolution

Traditional optimization-based blind deconvolution methods can be further categorized into two groups, i.e., Variational Bayes (VB)-based and MAP-based methods. VB-based method [21] is theoretically promising, but is with heavy computational cost. As for the MAP-based methods, many priors have been suggested for modeling clean image and blur kernels. In the seminal work of [2], Chan et al. introduce the total variation (TV) regularization to model latent clean image in blind deconvolution, and motivates several variants based on gradient-based priors, e.g., 0-norm [28] and p-norm [56]. Other specifically designed regularizations, e.g., 1/2-norm [16], patch-based prior [41, 25], low-rank prior [34] and dark channel prior [30, 50] have also been proposed to identify and preserve salient edges for benefiting blur kernel estimation. Recently, a discriminative prior [22] is presented to distinguish the clean image from a blurry one, but still heavily relies on 0-norm regularizer for attaining state-of-the-art performance. As for Pr(𝐤), gradient sparsity priors [56, 30, 20] and spectral prior [23] are usually adopted. In order to solve the MAP-based model, several tricks have been introduced to the projected alternating minimization algorithm, including delayed normalization of blur kernel [33], multi-scale implementation [16] and time-varying parameters [56].

After blur kernel estimation, non-blind deconvolution is required to recover the latent clean image with fine texture details [16, 28, 30, 41, 49, 23]. Thus, the priors should favor natural images, e.g., hyper-Laplacian [15], GMM [55], non-local similarity [5], e.g., RTF [37], CSF [36] and CNN [17, 53], which are quite different from those used in blur kernel estimation. Our SelfDeblur can be regarded as a special MAP-based method, but two generative networks, i.e., DIP and FCN, are adopted to respectively capture the deep priors of clean image and blur kernel. Moreover, the joint optimization algorithm is effective to estimate blur kernel and generate clean image, making non-blind deconvolution not a compulsory choice for SelfDeblur.

2.2 Deep Learning in Image Deblurring

Many studies have been given to apply deep learning (DL) to blind deblurring. For example, DL can be used to help the learning of mapping to blur kernel. By imitating the alternating minimization steps in optimization-based methods, Schuler et al. [38] design the deep network architectures for blur kernel estimation. By studying the spectral property of blurry images, deep CNN is suggested to predict the Fourier coefficients [1], which can then be projected to estimate blur kernel. In [40], CNN is used to predict the parametric blur kernels for motion blurry images.

For dynamic scene deblurring, deep CNNs have been developed to learn the direct mapping to latent clean image [51, 26, 43, 40, 18]. Motivated by the multi-scale strategy in blind deconvolution, multi-scale CNN [26] and scale-recurrent network [43] are proposed to directly estimate the latent clean image from the blurry image. The adversarial loss is also introduced for better recovery of texture details in motion deblurring [18]. Besides, by exploiting the temporal information between adjacent frames, deep networks have also been applied to video motion deblurring [27, 10, 31]. However, due to the severe ill-posedness caused by large size and complex blur kernels, existing DL-based methods still cannot outperform traditional optimization-based ones for blind deconvolution.

Recently, DIP [45] and Double-DIP [6] have been introduced to capture image statistics, and have been deployed to many low-level vision tasks such as super-resolution, inpainting, dehazing, transparency separation, etc. Nonetheless, the DIP network is limited in capturing the prior of blur kernels, and Double-DIP still performs poorly for blind deconvolution. To the best of our knowledge, our SelfDeblur makes the first attempt of applying deep networks to yield state-of-the-art blind deconvolution performance.

3 Proposed Method

In this section, we first introduce the general formulation of MAP-based blind deconvolution, and then present our proposed neural blind deconvolution model as well as the joint optimization algorithm.

3.1 MAP-based Blind Deconvolution Formulation

According to Eqn. (1), we define the fidelity term as -log(Pr(𝐲|𝐤,𝐱))=𝐤𝐱-𝐲2. And we further introduce two regularization terms -log(Pr(𝐱))=ϕ(𝐱) and -log(Pr(𝐤))=φ(𝐤) for modeling the priors on latent clean image and blur kernel, respectively. The MAP-based blind deconvolution model in Eqn. (2) can then be reformulated as,

( 𝐱,𝐤)=argmin(𝐱,𝐤)𝐤𝐱-𝐲2+λϕ(𝐱)+τφ(𝐤) (3)
s.t. 0xi1,i,kj0,jkj=1,j,

where λ and τ are trade-off regularization parameters. Besides the two regularization terms, we further introduce the non-negative and equality constraints for blur kernel 𝐤 [41, 56, 30, 33], and the pixels in 𝐱 are also constrained to the range [0,1].

Under the MAP-based framework, many fixed and handcrafted regularization terms have been presented for latent clean image and blur kernel [56, 30, 23, 41, 25, 34, 30, 50]. To solve the model in Eqn. (3), projected alternating minimization is generally adopted, but several optimization details, e.g., delayed normalization [33] and multi-scale implementation [16], are also crucial to the success of blind deconvolution. Moreover, once the estimated blur kernel 𝐤^ is obtained by solving Eqn. (3), another non-blind deconvolution usually is required to generate final deblurring result,

𝐱=argmin𝐱𝐤^𝐱-𝐲2+λ(𝐱), (4)

where (𝐱) is a regularizer to capture natural image statistics and is quite different from ϕ(𝐱).

3.2 Neural Blind Deconvolution

Motivated by the success of DIP [45] and Double-DIP [6], we suggest the neural blind deconvolution model by adopting generative networks 𝒢x and 𝒢k to capture the priors of 𝐱 and 𝐤. By substituting 𝐱 and 𝐤 with 𝒢x and 𝒢k and removing the regularization terms ϕ(𝐱) and φ(𝐤), the neural blind deconvolution can be formulated as,

min(𝒢x,𝒢k)𝒢k(𝐳k)𝒢x(𝐳x)-𝐲2 (5)
s.t.  0(𝒢x(𝐳x))i1,i,
(𝒢k(𝐳k))j0,j(𝒢k(𝐳k))j=1,j,

where 𝐳x and 𝐳k are sampled from the uniform distribution, ()i and ()j denote the i-th and j-th elements.

However, there remain several issues to be addressed with neural blind deconvolution. (i) The DIP network [45] is designed to capture low level image statistics and is limited in capturing the prior of blur kernels. As a result, we empirically find that Double-DIP [6] performs poorly for blind deconvolution (see the results in Sec. 4.1.2). (ii) Due to the non-negative and equality constraints, the resulting model in Eqn. (5) is a constrained neural optimization problem and is difficult to optimize. (iii) Although the generative networks 𝒢x and 𝒢k present high impedance to image noise, the denoising performance of DIP heavily relies on the additional averaging over last iterations and different optimization runs [45]. Such heuristic solutions, however, both bring more computational cost and cannot be directly borrowed to handle blurry and noisy images.

In the following, we present our solution to address the issues (i)&(ii) by designing proper generative networks 𝒢x and 𝒢k. As for the issue (iii), we introduce an extra TV regularizer and a regularization parameter to explicitly consider noise level in the neural blind deconvolution model.

Generative Network Gx. The latent clean images usually contain salient structures and rich textures, which requires the generative network 𝒢x to have sufficient modeling capacity. Fortunately, since the introduction of generative adversarial network [8], dramatic progress has been made in generating high quality natural images [45]. For modeling 𝐱, we adopt a DIP network, i.e., the asymmetric Autoencoder [35] with skip connections in [45], to serve as 𝒢x. As shown in Fig. 2, the first 5 layers of encoder are skip connected to the last 5 layers of decoder. Finally, a convolutional output layer is used to generate latent clean image. To meet the range constraint for 𝐱, the Sigmoid nonlinearity is applied to the output layer. Please refer to the supplementary material for more architecture details of 𝒢x.

Generative Network Gk. On the one hand, the DIP network [45] is designed to capture the statistics of natural image but performs limited in modeling the prior of blur kernel. On the other hand, blur kernel 𝐤 generally contains much fewer information than latent clean image 𝐱, and can be well generated by simpler generative network. Thus, we simply adopt a fully-connected network (FCN) to serve as 𝒢k. As shown in Fig. 2, the FCN 𝒢k takes a 1D noise 𝐳k with 200 dimensions as input, and has a hidden layer of 1,000 nodes and an output layer of K2 nodes. To guarantee the non-negative and equalitly constraints can be always satisfied, the SoftMax nonlinearity is applied to the output layer of 𝒢k. Finally, the 1D output of K2 entries is reshaped to a 2D K×K blur kernel. Please refer to the supplementary material for more architecture details of 𝒢k.

Unconstrained Neural Blind Deconvolution with TV Regularization. With the above generative networks 𝒢x and 𝒢k, we can formulate neural blind deconvolution into an unconstrained optimization form. However, the resulting model is irrelevant with the noise level, making it perform poorly on blurry images with non-negligible noise. To address this issue, we combine both 𝒢x and TV regularization to capture image priors, and our neural blind deconvolution model can then be written as,

min𝒢k,𝒢x𝒢k(𝐳k)𝒢x(𝐳x)-𝐲2+λTV(𝒢x(𝐳x)). (6)

where λ denotes the regularization parameter controlled by noise level σ. Albeit the generative network 𝒢x is more powerful, the incorporation of 𝒢x and another image prior generally is beneficial to deconvolution performance. Moreover, the introduction of the noise level related regularization parameter λ can greatly improve the robustness in handling blurry images with various noise levels. In particular, we emperically set λ=0.1×σ in our implementation, and the noise level σ can be estimated using the method introduced in [54].

3.3 Optimization Algorithm

The optimization process of Eqn. (6) can be explained as a kind of ”zero-shot” self-supervised learning [39], where the generative networks 𝒢k and 𝒢x are trained using only a test image (i.e., blurry image 𝐲) and no ground-truth clean image is available. Thus, our method is dubbed SelfDeblur. In the following, we present two algorithms for SelfDeblur, i.e., alternating optimization and joint optimization.

Alternating Optimization. Analogous to the alternating minimization steps in traditional blind deconvolution [56, 2, 30, 28, 41], the network parameters of 𝒢k and 𝒢x can also be optimized in an alternating manner. As summarized in Algorithm 3.3, the parameters of 𝒢k are updated via the ADAM [14] by fixing 𝒢x, and vice versa. In particular, the gradient w.r.t. either 𝒢x or 𝒢k can be derived using automatic differentiation [32].

Joint Optimization. In traditional MAP-based framework, alternating minimization allows the use of projection operator to handle non-negative and equality constraints and the modification of optimization details to avoid trivial solution, and thus has been widely adopted. As for our neural blind deconvolution, the model in Eqn. (6) is unconstrained optimization, and the powerful modeling capacity of 𝒢k and 𝒢x is beneficial to avoid trivial delta kernel solution. We also note that the unconstrained neural blind deconvolution is highly non-convex, and alternating optimization may get stuck at saddle points [44]. Thus, joint optimization is more prefered than alternating optimization for SelfDeblur. Using the automatic differentiation techniques [32], the gradients w.r.t. 𝒢k and 𝒢x can be derived. Algorithm 3.3 summarizes the joint optimization algorithm, where the parameters of 𝒢k and 𝒢x can be jointly updated using the ADAM algorithm. Our empirical study in Sec. 4.1.1 also shows that joint optimization usually converges to better solutions than alternating optimization.

Both alternating optimization and joint optimization algorithms are stopped when reaching T iterations. Then, the estimated blur kernel and latent clean image can be generated using 𝐤=𝒢kT(𝐳k) and 𝐱=𝒢xT(𝐳x), respectively. Benefited from the modeling capacity of 𝒢x(𝐳x), the estimated 𝐱 is with visually favorable textures, and it is not a compulsory choice for our SelfDeblur to adopt another non-blind deconvolution method to generate final deblurring result.

{algorithm}

[!t] SelfDeblur (Alternating Optimization) \[email protected]@algorithmic[1] \RequireBlurry image 𝐲 \EnsureBlur kernel 𝐤 and clean image 𝐱 \StateSample 𝐳x and 𝐳k from uniform distribution with seed 0. \State𝐤=𝒢k0(𝐳k) \Fort=1 to T

\State

𝐱=𝒢xt-1(𝐳x) \StateCompute the gradient w.r.t. 𝒢k \StateUpdate 𝒢kt using the ADAM algorithm [14] \State𝐤=𝒢kt(𝐳k) \StateCompute the gradient w.r.t. 𝒢x \StateUpdate 𝒢xt using the ADAM algorithm [14]

\EndFor\State

𝐱=𝒢xT(𝐳x), 𝐤=𝒢xT(𝐳k)

{algorithm}

[!ht] SelfDeblur (Joint Optimization) \[email protected]@algorithmic[1] \RequireBlurry image 𝐲 \EnsureBlur kernel 𝐤 and clean image 𝐱 \StateSample 𝐳x and 𝐳k from uniform distribution with seed 0. \Fort=1 to T

\State

𝐤=𝒢kt-1(𝐳k) \State𝐱=𝒢xt-1(𝐳x) \StateCompute the gradients w.r.t. 𝒢k and 𝒢x \StateUpdate 𝒢kt and 𝒢xt using the ADAM algorithm [14]

\EndFor\State

𝐱=𝒢xT(𝐳x), 𝐤=𝒢xT(𝐳k)

4 Experimental Results

In this section, ablation study is first conducted to analyze the effect of optimization algorithm and network architecture. Then, our SelfDeblur is evaluated on two benchmark datasets and is compared with the state-of-the-art blind deconvolution methods. Finally, we report the results of SelfDeblur on several real-world blurry images.

Our SelfDeblur is implemented using Pytorch [32]. The experiments are conducted on a PC equipped with one NVIDIA Titan V GPU. Unless specially stated, the experiments follow the same settings, i.e., T=5,000, and the noises 𝐳x and 𝐳k are sampled from the uniform distribution with fixed random seed 0. Following [45], we further perturb 𝐳x randomly at each iteration. The initial learning rate is set as 0.01 and is decayed by multiplying 0.5 when reaching 2,000, 3,000 and 4,000 iterations. The source code, deep models and supplementary materials are available at https://github.com/csdwren/SelfDeblur.

4.1 Ablation Study

Ablation study is conducted on the dataset by Levin et al. [20], which is a popular blind deconvolution benchmark consisting of 4 clean images and 8 blur kernels. Using [54], the average estimated noise level of the blurry images in the dataset is σ1×10-5. Thus we simply adopt λ=1×10-6 on this dataset.

4.1.1 Alternating Optimization vs. Joint Optimization

We first evaluate the performance of SelfDeblur using alternating optimization (SelfDeblur-A) and joint optimization (SelfDeblur-J). Table 1 reports the average PSNR and SSIM values. In terms of quantitative metrics, SelfDeblur-J significantly outperforms SelfDeblur-A, demonstrating the superiority of joint optimization. Fig. 3 provides several failure cases of SelfDeblur-A. It can be seen that SelfDeblur-A may converge to delta kernel and worse solution, while SelfDeblur-J performs favorably on these cases. Therefore, joint optimization is adopted as the default SelfDeblur method throughout the following experiments.

Blurry images SelfDeblur-A SelfDeblur-J
Figure 3: Failure cases of SelfDeblur-A.
Table 1: Average PSNR/SSIM comparison of SelfDeblur-A and SelfDeblur-J on the dataset of Levin et al. [20].
SelfDeblur-A SelfDeblur-J
30.53 / 0.8748 33.07 / 0.9313

4.1.2 Network Architecture of 𝒢k

In this experiment, we compare the results by considering four kinds of network architectures: (i) SelfDeblur, (ii) Double-DIP [6] (asymmetric Autoencoder with skip connections for both 𝒢x and 𝒢k), (iii) SelfDeblurk- (removing the hidden layer from 𝒢k), and (iv) SelfDeblurk+ (adding an extra hidden layer for 𝒢k). From Table 2 and Fig. 4, SelfDeblur significantly outperforms Double-DIP in estimating blur kernel and latent image. The result indicates that the DIP network is limited to capture the prior of blur kernel, and the simple FCN can be a good choice of 𝒢k. We further compare SelfDeblur with SelfDeblurk- and SelfDeblurk+. One can see that the FCN without hidden layer (i.e., SelfDeblurk-) also succeeds in estimating blur kernel and clean image (see Fig. 4), but performs much inferior to SelfDeblur. Moreover, the three-layer FCN ()i.e., SelfDeblurk+) is superior to SelfDeblurk-, but is inferior to SelfDeblur. To sum up, SelfDeblur is a good choice for modeling blur kernel prior.

Table 2: Quantitavie comparison of SelfDeblur variants with different network structures of 𝒢k.
SelfDeblur SelfDeblurk- SelfDeblurk+ Double-DIP
PSNR 33.07 28.37 30.92 21.51
SSIM 0.9313 0.8396 0.8889 0.5256
SelfDeblur SelfDeblurk- SelfDeblurk+ Double-DIP
Figure 4: Visual comparison of SelfDeblur variants with different network structures of 𝒢k.

4.1.3 Visualization of Intermediate Results

Using an image from the dataset of Levin et al[20], Fig. 5 shows the intermediate results of estimated blur kernel and clean image at iteration t = 1,20,100,600,2,000 and 5,000, along with the MSE curve for 𝐤 and the PSNR curve for 𝐱. When iteration t=20, the intermediate result of 𝐱 mainly contains the salient image structures, which is consistent with the observation that salient edges is crucial for initial blur kernel estimation in traditional methods. Along with the increase of iterations, 𝒢x and 𝒢k begin to generate finer details in 𝐱 and 𝐤. Unlike traditional methods, SelfDeblur is effective in simultaneously estimating blur kernel and recover latent clean image when iteration t20, making the non-blind deconvolution not a compulsory choice for SelfDeblur.

Figure 5: Intermediate results of estimated blur kernel and latent clean image at iteration t=1,20,100,600,2,000 and 5,000.

4.2 Comparison with State-of-the-arts

4.2.1 Results on dataset of Levin et al. [20]

Using the dataset of Levin et al. [20], we compare our SelfDeblur with several state-of-the-art blind deconvolution methods, including Krishnan et al. [16], Levin et al. [20], Cho&Lee [4], Xu&Jia [48], Sun et al. [41], Zuo et al. [56] and Pan-DCP [30]. For SelfDeblur, λ=1×10-6 is set for all the blurry images. Following [41, 56], we adopt the non-blind deconvolution method in [21] to generate final deblurring results. PSNR, SSIM [46] and Error Ratio [21] are used as quantitative metrics. And we also report the running time (sec.) of blur kernel estimation for each competing method. Our SelfDeblur is ran on an NVIDIA Titan V GPU, while the other methods are ran on a PC with 3.30GHz Intel(R) Xeon(R) CPU.

Blurry image Zuo et al. Δ[56] Xu&JiaΔ[48] SelfDeblurΔ
Ground-truth Sun et al. Δ[41] Pan-DCPΔ[30] SelfDeblur
Figure 6: Visual comparison on the dataset of Levin et al. [20].
Table 3: Average PSNR, SSIM, Error Ratio and running time (sec.) comparison on the dataset of Levin et al. [20]. Here, Δ indicates the method generates final deblurring results using the non-blind deconvolution method from [21].
PSNR SSIM Error Ratio Time
Known kΔ 34.53 0.9492 1.0000
Krishnan et al. Δ [16] 29.88 0.8666 2.4523 8.9400
Cho&LeeΔ [4] 30.57 0.8966 1.7113 1.3951
Levin et al. Δ [21] 30.80 0.9092 1.7724 78.263
Xu&JiaΔ [49] 31.67 0.9163 1.4898 1.1840
Sun et al. Δ [41] 32.99 0.9330 1.2847 191.03
Zuo et al. Δ [56] 32.66 0.9332 1.2500 10.998
Pan-DCPΔ [30] 32.69 0.9284 1.2555 295.23
SelfDeblurΔ 33.32 0.9438 1.2509
SelfDeblur 33.07 0.9313 1.1968 224.01
Table 4: Comparison of the blur kernel estimation performance on the dataset of Levin et al.
Krishnan et al. [16] Levin et al. [21] Cho&Lee [4] Xu&Jia [48] Sun et al. [41] Zuo et al. [56] Pan-DCP [30] SelfDeblur
MSE 675.4 307.0 340.8 402.1 254.7 351.9 347.2 149.5
MNC[9] 0.8261 0.8693 0.8635 0.8811 0.9305 0.9042 0.8936 0.9408
Table 5: Average PSNR/SSIM comparison on the dataset of Lai et al. [19], which has 5 categories. The methods marked with Δ adopt [47] and [15] as non-blind deconvolution after blur kernel estimation in Saturated and the other categories, respectively.
Images Cho&LeeΔ[4] Xu&JiaΔ[48] Xu et al. Δ[49] Michaeli et al. Δ[25] Perroe et al. Δ [33] Pan-L0Δ[28] Pan-DCPΔ[30] SelfDeblurΔ SelfDeblur
Manmade 16.35/0.3890 19.23/0.6540 17.99/0.5986 17.43/0.4189 17.41/0.5507 16.92/0.5316 18.59/0.5942 20.08/0.7338 20.35/0.7543
Natural 20.14/0.5198 23.03/0.7542 21.58/0.6788 20.70/0.5116 21.04/0.6764 20.92/0.6622 22.60/0.6984 22.50/0.7183 22.05/0.7092
People 19.90/0.5560 25.32/0.8517 24.40/0.8133 23.35/0.6999 22.77/0.7347 23.36/0.7822 24.03/0.7719 27.41/0.8784 25.94/0.8834
Saturated 14.05/0.4927 14.79/0.5632 14.53/0.5383 14.14/0.4914 14.24/0.5107 14.62/0.5451 16.52/0.6322 16.58/0.6165 16.35/0.6364
Text 14.87/0.4429 18.56/0.7171 17.64/0.6677 16.23/0.4686 16.94/0.5927 16.87/0.6030 17.42/0.6193 19.06/0.7126 20.16/0.7785
Avg. 17.06/0.4801 20.18/0.7080 19.23/0.6593 18.37/0.5181 18.48/0.6130 18.54/0.6248 19.89/0.6656 21.13/0.7319 20.97/0.7524
Figure 7: Average PSNR and SSIM comparison for each blur kernel on the dataset of Levin et al. [20]. For each kernel, the methods from left to right are Krishnan et al. Δ[16], Cho&LeeΔ[4], Levin et al. Δ[20], Xu&JiaΔ[48], Sun et al. Δ[41], Pan-DCPΔ[30], SelfDeblurΔ and SelfDeblur.

Table 3 lists the average metrics of the competing methods. We report the results of SelfDeblur with two settings, i.e., the deblurring results purely by SelfDeblur and those using the non-blind deconvolution from [21], denoted as SelfDeblurΔ. In terms of PSNR and Error Ratio, SelfDeblur significantly outperforms the competing methods. As for average SSIM, SelfDeblur performs slightly inferior to Sun et al. and Zuo et al. By incorporating with non-blind deconvolution from [21], SelfDeblurΔ can further boost quantitative performance and outperforms all the other methods. In terms of running time, SelfDeblur is time-consuming due to the optimization of two generative networks, but is comparable with Sun et al. [41] and Pan-DCP [30].

We further provide the average PSNR and SSIM values for each blur kernel in Fig. 7. From the visual comparison in Fig. 6, the #4 blur kernel estimated by SelfDeblur is much closer to the ground-truth. As shown in the close-ups, SelfDeblur and SelfDeblurΔ can recover more visually favorable textures. We also note that both the performance gap and visual quality between SelfDeblur and SelfDeblurΔ are not significant, and thus non-blind deconvolution is not a compulsory choice for our SelfDeblur.

Moreover, we evaluate the estimated kernels using two metrics, i.e., MSE best aligned to ground-truth blur kernel 𝐤gt and maximum of normalized convolution (MNC) [9],

MNC=max(𝐤𝐤gt𝐤2𝐤gt2).

From Table 4, our SelfDeblur is much superior to the competing methods in terms of both the metrics.

4.2.2 Results on dataset of Lai et al. [19]

We further evaluate SelfDeblur on the dataset of Lai et al. [19] [19] consisting of 25 clean images and 4 large size blur kernels. The blurry images are divided into five categories, i.e., Manmade, Natural, People, Saturated and Text, where each category contains 20 blurry images. For each blurry image, the parameter λ is set according to the noise level estimated using [54]. We compare our SelfDeblur with Cho&Lee [4], Xu&Jia[48], Xu et al. [49], Machaeli et al. [25], Perroe et al. [33], Pan-L0 [28] and Pan-DCP [30]. The results of competing methods except Pan-DCP [30] and ours are duplicated from [19]. The results of Pan-DCP [30] are generated using their default settings. Once the estimated blur kernel is obtained, non-blind deconvolution [15] is applied to the images of Manmade, Natural, People and Text, while [47] is used to handle Saturated images.

Blurry image Xu&JiaΔ[48] Perrone et al. Δ[33] SelfDeblurΔ
Ground-truth Michaeli et al. Δ[48] Pan-DCPΔ[33] SelfDeblur
Figure 8: Visual comparison on the dataset of Lai et al. [19].
Figure 9: Average PSNR and SSIM comparison for each blur kernel on the dataset of Lai et al. [19]. For each kernel, the methods from left to right are Cho&LeeΔ[4], Xu&JiaΔ[48], Michaeli et al. Δ[25], Perroe et al. Δ[33], Pan-L0Δ[28], Pan-DCPΔ[30], SelfDeblurΔ and SelfDeblur.
Blurry image Xu&Jia[48] Pan-DCP[30] SelfDeblur
Figure 10: Visual comparison on two real-world blurry images.

From Table 5, both SelfDeblur and SelfDeblurΔ can achieve better quantitative metrics than the competing methods. In terms of image contents, our SelfDeblur outperform the other methods on any of the five categories. Fig. 9 shows the average PSNR and SSIM values for each blur kernel. From the results in Fig. 8, the blur kernel estimated by our SelfDeblur is more accurate than those by the competing methods, and the deconvolution result is with more visually plausible textures.

4.3 Evaluation on Real-world Blurry Images

Our SelfDeblur is further compared with Xu&Jia [48] and Pan-DCP [30] on several real-world blurry images. From Fig. 10, one can see that the blur kernels estimated by our SelfDeblur contain less noises, and the estimated clean images are with more visually plausible structures and textures. The kernel estimation errors by Xu&Jia and Pan-DCP are obvious, thereby yielding ringing artifacts in the estimated clean images. More results can be found in the supplementary materials.

5 Conclusion

In this paper, we proposed a neural blind deconvolution method, i.e., SelfDeblur. It adopts an asymmetric Autoencoder and a FCN to respectively capture the deep priors of latent clean image and blur kernel. And the SoftMax nonlinearity is applied to the output of FCN to meet the non-negative and equality constraints of blur kernel. A joint optimization algorithm is suggested to solve the unconstrained neural blind deconvolution model. Experiments show that our SelfDeblur achieves notable performance gains over the state-of-the-art methods, and is effective in estimating blur kernel and generating clean image with visually favorable textures.

References

  • [1] A. Chakrabarti (2016) A neural approach to blind motion deblurring. In European Conference on Computer Vision, pp. 221–235. Cited by: §1, §2.2.
  • [2] T. F. Chan and C. Wong (1998) Total variation blind deconvolution. IEEE Transactions on Image Processing 7 (3), pp. 370–375. Cited by: §1, §1, §2.1, §3.3.
  • [3] L. Chen, F. Fang, T. Wang, and G. Zhang (2019) Blind image deblurring with local maximum gradient prior. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1742–1750. Cited by: §1.
  • [4] S. Cho and S. Lee (2009) Fast motion deblurring. ACM Transactions on Graphics 28 (5), pp. 145. Cited by: §1, Figure 7, Figure 9, §4.2.1, §4.2.2, Table 3, Table 4, Table 5.
  • [5] W. Dong, L. Zhang, G. Shi, and X. Li (2013) Nonlocally centralized sparse representation for image restoration. IEEE Transactions on Image Processing 22 (4), pp. 1620–1630. Cited by: §2.1.
  • [6] Y. Gandelsman, A. Shocher, and M. Irani (2019) ” Double-dip”: unsupervised image decomposition via coupled deep-image-priors. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Cited by: §1, §2.2, §3.2, §3.2, §4.1.2.
  • [7] H. Gao, X. Tao, X. Shen, and J. Jia (2019) Dynamic scene deblurring with parameter selective sharing and nested skip connections. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3848–3856. Cited by: §1.
  • [8] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio (2014) Generative adversarial nets. In Advances in Neural Information Processing Systems, pp. 2672–2680. Cited by: §3.2.
  • [9] Z. Hu and M. Yang (2012) Good regions to deblur. In European conference on computer vision, pp. 59–72. Cited by: §4.2.1, Table 4.
  • [10] T. Hyun Kim, K. Mu Lee, B. Scholkopf, and M. Hirsch (2017) Online video deblurring via dynamic temporal blending network. In Proceedings of the IEEE International Conference on Computer Vision, pp. 4038–4047. Cited by: §2.2.
  • [11] M. Jin, S. Roth, and P. Favaro (2018) Normalized blind deconvolution. In European Conference on Computer Vision, Cited by: §1.
  • [12] J. Kim, J. Kwon Lee, and K. Mu Lee (2016) Accurate image super-resolution using very deep convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1646–1654. Cited by: §1.
  • [13] J. Kim, J. Kwon Lee, and K. Mu Lee (2016) Deeply-recursive convolutional network for image super-resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1637–1645. Cited by: §1.
  • [14] D. P. Kingma and J. Ba (2015) Adam: a method for stochastic optimization. In International Conference on Learning Representations, Cited by: §3.3, §3.3, §3.3.
  • [15] D. Krishnan and R. Fergus (2009) Fast image deconvolution using hyper-Laplacian priors. In Advances in Neural Information Processing Systems, pp. 1033–1041. Cited by: §2.1, §4.2.2, Table 5.
  • [16] D. Krishnan, T. Tay, and R. Fergus (2011) Blind deconvolution using a normalized sparsity measure. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 233–240. Cited by: §1, §2.1, §2.1, §3.1, Figure 7, §4.2.1, Table 3, Table 4.
  • [17] J. Kruse, C. Rother, and U. Schmidt (2017) Learning to push the limits of efficient fft-based image deconvolution. In Proceedings of the IEEE International Conference on Computer Vision, pp. 4586–4594. Cited by: §2.1.
  • [18] O. Kupyn, V. Budzan, M. Mykhailych, D. Mishkin, and J. Matas (2018) Deblurgan: blind motion deblurring using conditional adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8183–8192. Cited by: §2.2.
  • [19] W. Lai, J. Huang, Z. Hu, N. Ahuja, and M. Yang (2016) A comparative study for single image blind deblurring. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1701–1709. Cited by: §1, Figure 8, Figure 9, §4.2.2, §4.2.2, Table 5.
  • [20] A. Levin, Y. Weiss, F. Durand, and W. T. Freeman (2009) Understanding and evaluating blind deconvolution algorithms. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1964–1971. Cited by: §1, §1, §1, §2.1, Figure 6, Figure 7, §4.1.3, §4.1, §4.2.1, §4.2.1, Table 1, Table 3.
  • [21] A. Levin, Y. Weiss, F. Durand, and W. T. Freeman (2011) Efficient marginal likelihood optimization in blind deconvolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2657–2664. Cited by: §2.1, §4.2.1, §4.2.1, Table 3, Table 4.
  • [22] L. Li, J. Pan, W. Lai, C. Gao, N. Sang, and M. Yang (2018) Learning a discriminative prior for blind image deblurring. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6616–6625. Cited by: §2.1.
  • [23] G. Liu, S. Chang, and Y. Ma (2014) Blind image deblurring using spectral properties of convolution operators. IEEE Transactions on Image Processing 23 (12), pp. 5047–5056. Cited by: §1, §2.1, §2.1, §3.1.
  • [24] R. Liu, S. Cheng, Y. He, X. Fan, Z. Lin, and Z. Luo (2019) On the convergence of learning-based iterative methods for nonconvex inverse problems. IEEE Transactions on Pattern Analysis and Machine Intelligence. Cited by: §1.
  • [25] T. Michaeli and M. Irani (2014) Blind deblurring using internal patch recurrence. In European Conference on Computer Vision, pp. 783–798. Cited by: §1, §2.1, §3.1, Figure 9, §4.2.2, Table 5.
  • [26] S. Nah, T. H. Kim, and K. M. Lee (2017) Deep multi-scale convolutional neural network for dynamic scene deblurring. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 1, pp. 3. Cited by: §1, §2.2.
  • [27] S. Nah, S. Son, and K. M. Lee (2019) Recurrent neural networks with intra-frame iterations for video deblurring. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8102–8111. Cited by: §2.2.
  • [28] J. Pan, Z. Hu, Z. Su, and M. Yang (2017) L0-Regularized intensity and gradient prior for deblurring text images and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence 39 (2), pp. 342–355. Cited by: Figure 1, §1, §2.1, §2.1, §3.3, Figure 9, §4.2.2, Table 5.
  • [29] J. Pan, W. Ren, Z. Hu, and M. Yang (2018) Learning to deblur images with exemplars. IEEE Transactions on Pattern Analysis and Machine Intelligence. Cited by: §1.
  • [30] J. Pan, D. Sun, H. Pfister, and M. Yang (2018) Deblurring images via dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence 40 (10), pp. 2315–2328. Cited by: Figure 1, §1, §1, §2.1, §2.1, §3.1, §3.1, §3.3, Figure 10, Figure 6, Figure 7, Figure 9, §4.2.1, §4.2.1, §4.2.2, §4.3, Table 3, Table 4, Table 5.
  • [31] L. Pan, Y. Dai, M. Liu, and F. Porikli (2017) Simultaneous stereo video deblurring and scene flow estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4382–4391. Cited by: §2.2.
  • [32] A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer (2017) Automatic differentiation in pytorch. In NIPS Autodiff Workshop: The Future of Gradient-based Machine Learning Software and Techniques, Cited by: §3.3, §3.3, §4.
  • [33] D. Perrone and P. Favaro (2014) Total variation blind deconvolution: the devil is in the details. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2909–2916. Cited by: §1, §1, §2.1, §3.1, §3.1, Figure 8, Figure 9, §4.2.2, Table 5.
  • [34] W. Ren, X. Cao, J. Pan, X. Guo, W. Zuo, and M. Yang (2016) Image deblurring via enhanced low-rank prior. IEEE Transactions on Image Processing 25 (7), pp. 3426–3437. Cited by: §1, §2.1, §3.1.
  • [35] O. Ronneberger, P. Fischer, and T. Brox (2015) U-net: convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-assisted Intervention, pp. 234–241. Cited by: §3.2.
  • [36] U. Schmidt and S. Roth (2014) Shrinkage fields for effective image restoration. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2774–2781. Cited by: §2.1.
  • [37] U. Schmidt, C. Rother, S. Nowozin, J. Jancsary, and S. Roth (2013) Discriminative non-blind deblurring. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 604–611. Cited by: §2.1.
  • [38] C. J. Schuler, M. Hirsch, S. Harmeling, and B. Schölkopf (2016) Learning to deblur. IEEE Transactions on Pattern Analysis and Machine Intelligence 38 (7), pp. 1439–1451. Cited by: §1, §2.2.
  • [39] A. Shocher, N. Cohen, and M. Irani (2018) Zero-shot super-resolution using deep internal learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3118–3126. Cited by: §1, §3.3.
  • [40] J. Sun, W. Cao, Z. Xu, and J. Ponce (2015) Learning a convolutional neural network for non-uniform motion blur removal. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 769–777. Cited by: §2.2, §2.2.
  • [41] L. Sun, S. Cho, J. Wang, and J. Hays (2013) Edge-based blur kernel estimation using patch priors. In IEEE International Conference on Computational Photography, pp. 1–8. Cited by: Figure 1, §1, §2.1, §2.1, §3.1, §3.1, §3.3, Figure 6, Figure 7, §4.2.1, §4.2.1, Table 3, Table 4.
  • [42] Y. Tai, J. Yang, and X. Liu (2017) Image super-resolution via deep recursive residual network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 1, pp. 5. Cited by: §1.
  • [43] X. Tao, H. Gao, X. Shen, J. Wang, and J. Jia (2018) Scale-recurrent network for deep image deblurring. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8174–8182. Cited by: §1, §2.2.
  • [44] P. Tseng (2001) Convergence of a block coordinate descent method for nondifferentiable minimization. Journal of optimization theory and applications 109 (3), pp. 475–494. Cited by: §3.3.
  • [45] D. Ulyanov, A. Vedaldi, and V. Lempitsky (2018) Deep image prior. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9446–9454. Cited by: §1, §1, §2.2, §3.2, §3.2, §3.2, §3.2, §4.
  • [46] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli (2004) Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing 13 (4), pp. 600–612. Cited by: §4.2.1.
  • [47] O. Whyte, J. Sivic, and A. Zisserman (2014) Deblurring shaken and partially saturated images. International Journal of Computer Vision 110 (2), pp. 185–201. Cited by: §4.2.2, Table 5.
  • [48] L. Xu and J. Jia (2010) Two-phase kernel estimation for robust motion deblurring. In European Conference on Computer Vision, pp. 157–170. Cited by: Figure 1, Figure 10, Figure 6, Figure 7, Figure 8, Figure 9, §4.2.1, §4.2.2, §4.3, Table 4, Table 5.
  • [49] L. Xu, S. Zheng, and J. Jia (2013) Unnatural l0 sparse representation for natural image deblurring. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1107–1114. Cited by: §2.1, §4.2.2, Table 3, Table 5.
  • [50] Y. Yan, W. Ren, Y. Guo, R. Wang, and X. Cao (2017) Image deblurring via extreme channels prior. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4003–4011. Cited by: §1, §2.1, §3.1.
  • [51] J. Zhang, J. Pan, J. Ren, Y. Song, L. Bao, R. W. Lau, and M. Yang (2018) Dynamic scene deblurring using spatially variant recurrent neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2521–2529. Cited by: §1, §2.2.
  • [52] K. Zhang, W. Zuo, Y. Chen, D. Meng, and L. Zhang (2017) Beyond a gaussian denoiser: residual learning of deep cnn for image denoising. IEEE Transactions on Image Processing 26 (7), pp. 3142–3155. Cited by: §1.
  • [53] K. Zhang, W. Zuo, S. Gu, and L. Zhang (2017) Learning deep CNN denoiser prior for image restoration. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Cited by: §2.1.
  • [54] D. Zoran and Y. Weiss (2009) Scale invariance and noise in natural images. In Proceedings of the IEEE International Conference on Computer Vision, pp. 2209–2216. Cited by: §3.2, §4.1, §4.2.2.
  • [55] D. Zoran and Y. Weiss (2011) From learning models of natural image patches to whole image restoration. In Proceedings of the IEEE International Conference on Computer Vision, pp. 479–486. Cited by: §2.1.
  • [56] W. Zuo, D. Ren, D. Zhang, S. Gu, and L. Zhang (2016) Learning iteration-wise generalized shrinkage–thresholding operators for blind deconvolution. IEEE Transactions on Image Processing 25 (4), pp. 1751–1764. Cited by: §1, §1, §2.1, §3.1, §3.1, §3.3, Figure 6, §4.2.1, Table 3, Table 4.