Journal of Computer Science and Technology ›› 2019, Vol. 34 ›› Issue (3): 609-621.doi: 10.1007/s11390-019-1930-z

Special Issue: Artificial Intelligence and Pattern Recognition; Computer Graphics and Multimedia

• Artificial Intelligence and Pattern Recognition • Previous Articles     Next Articles

Blind Image Deblurring via Adaptive Optimization with Flexible Sparse Structure Control

Ri-Sheng Liu, Member, ACM, IEEE, Cai-Sheng Mao, Zhi-Hui Wang, Hao-Jie Li*, Member, ACM, IEEE   

  1. International School of Information Science and Engineering, Dalian University of Technology, Dalian 116620, China;Key Laboratory for Ubiquitous Network and Service Software of Liaoning Province, Dalian 116620, China
  • Received:2018-10-19 Revised:2019-03-26 Online:2019-05-05 Published:2019-05-06
  • Contact: Hao-Jie Li E-mail:hjli@dlut.edu.cn
  • About author:Ri-Sheng Liu is currently an associate professor in the International School of Information Science and Engineering, Dalian University of Technology, Dalian. He received his B.S. and Ph.D. degrees both in mathematics from the Dalian University of Technology, Dalian, in 2007 and 2012 respectively. He was a visiting scholar in the Robotic Institute of Carnegie Mellon University, Pittsburgh, from 2010 to 2012. He served as a Hong Kong scholar research fellow at the Hong Kong Polytechnic University, Hong Kong, from 2016 to 2017. He is a member of ACM and IEEE. His research interests include machine learning, optimization, computer vision and multimedia.
  • Supported by:
    This work was partially supported by the National Natural Science Foundation of China under Grant Nos. 61672125 and 61772108.

Blind image deblurring is a long-standing ill-posed inverse problem which aims to recover a latent sharp image given only a blurry observation. So far, existing studies have designed many effective priors w.r.t. the latent image within the maximum a posteriori (MAP) framework in order to narrow down the solution space. These non-convex priors are always integrated into the final deblurring model, which makes the optimization challenging. However, due to unknown image distribution, complex kernel structure and non-uniform noises in real-world scenarios, it is indeed challenging to explicitly design a fixed prior for all cases. Thus we adopt the idea of adaptive optimization and propose the sparse structure control (SSC) for the latent image during the optimization process. In this paper, we only formulate the necessary optimization constraints in a lightweight MAP model with no priors. Then we develop an inexact projected gradient scheme to incorporate flexible SSC in MAP inference. Besides lp-norm based SSC in our previous work, we also train a group of denoising convolutional neural networks (CNNs) to learn the sparse image structure automatically from the training data under different noise levels, and we show that CNNs-based SSC can achieve similar results compared with lp-norm but are more robust to noise. Extensive experiments demonstrate that the proposed adaptive optimization scheme with two types of SSC achieves the state-of-the-art results on both synthetic data and real-world images.

Key words: blind image deblurring; convolutional neural network (CNN); non-convex optimization; sparse structure control (SSC);

[1] Pan J, Sun D, Pfister H, Yang M H. Blind image deblurring using dark channel prior. In Proc. the 29th IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2016, pp.1628-1636.
[2] Chan T F, Wong C K. Total variation blind deconvolution. IEEE Transactions on Image Processing, 1998, 7(3):370- 375.
[3] Krishnan D, Fergus R. Fast image deconvolution using hyper-Laplacian priors. In Proc. the 23rd Annual Conference on Neural Information Processing Systems, Dec. 2009, pp.1033-1041.
[4] Levin A, Weiss Y, Durand F, Freeman W T. Efficient marginal likelihood optimization in blind deconvolution. In Proc. the 24th IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2011, pp.2657-2664.
[5] Sun L, Cho S, Wang J, Hays J. Edge-based blur kernel estimation using patch priors. In Proc. the 3rd IEEE International Conference on Computational Photography, Apr. 2013, Article No. 8.
[6] Pan J, Hu Z, Su Z, Yang M H. Deblurring text images via L0-regularized intensity and gradient prior. In Proc. the 27th IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2014, pp.2901-2908.
[7] Perrone D, Diethelm R, Favaro P. Blind deconvolution via lower-bounded logarithmic image priors. In Proc. the 10th International Workshop on Energy Minimization Methods in Computer Vision and Pattern Recognition, Jan. 2015, pp.112-125.
[8] Cho S, Lee S. Fast motion deblurring. ACM Transactions on Graphics, 2009, 28(5):Article No. 145.
[9] Krishnan D, Tay T, Fergus R. Blind deconvolution using a normalized sparsity measure. In Proc. the 24th IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2011, pp.233-240.
[10] Zuo W, Ren D, Zhang D, Gu S, Zhang L. Learning iterationwise generalized shrinkage-thresholding operators for blind deconvolution. IEEE Transactions on Image Processing, 2016, 25(4):1751-1764.
[11] Hu Z, Cho S, Wang J, Yang, M H. Deblurring low-light images with light streaks. In Proc. the 27th IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2014, pp.3382-3389.
[12] Mao C, Liu R, Li H. Adaptive optimization with nested prior navigation for blind image deblurring. In Proc. the 4th IEEE International Conference on Multimedia Big Data, Sept. 2018, Article No. 37.
[13] Xu L, Jia J. Two-phase kernel estimation for robust motion deblurring. In Proc. the 11th European Conference on Computer Vision, Sept. 2010, pp.157-170.
[14] Fergus R, Singh B, Hertzmann A, Roweis S T, Freeman W T. Removing camera shake from a single photograph. ACM Transactions on Graphics, 2006, 25(3):787-794.
[15] Levin A, Weiss Y, Durand F, Freeman W T. Understanding and evaluating blind deconvolution algorithms. In Proc. the 22nd IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2009, pp.1964-1971.
[16] Wipf D, Zhang H. Revisiting Bayesian blind deconvolution. The Journal of Machine Learning Research, 2014, 15(1):3595-3634.
[17] Harmeling S, Sra S, Hirsch M, Schölkopf B. Multiframe blind deconvolution, super-resolution, and saturation correction via incremental EM. In Proc. the 17th IEEE International Conference on Image Processing, Sept. 2010, pp.3313-3316.
[18] Gong D, Tan M, Zhang Y, van den Hengel A, Shi Q. Blind image deconvolution by automatic gradient activation. In Proc. the 29th IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2016, pp.1827-1836.
[19] Zhang K, Zuo W, Chen Y, Meng D, Zhang L. Beyond a Gaussian denoiser:Residual learning of deep CNN for image denoising. IEEE Transactions on Image Processing, 2017, 26(7):3142-3155.
[20] Ng M K, Chan R H, Tang W C. A fast algorithm for deblurring models with Neumann boundary conditions. SIAM Journal on Scientific Computing, 1999, 21(3):851-866.
[21] Donatelli M, Estatico C, Martinelli A, Serra-Capizzano S. Improved image deblurring with anti-reflective boundary conditions and re-blurring. Inverse Problems, 2006, 22(6):2035-2053.
[22] Liu R, Jia J. Reducing boundary artifacts in image deconvolution. In Proc. the 15th IEEE International Conference on Image Processing, Oct. 2008, pp.505-508.
[23] Reeves S J. Fast image restoration without boundary artifacts. IEEE Transactions on Image Processing, 2005, 14(10):1448-1453.
[24] Kruse J, Rother C, Schmidt U. Learning to push the limits of efficient FFT-based image deconvolution. In Proc. the 16th IEEE International Conference on Computer Vision, Oct. 2017, pp.4586-4594.
[25] Zuo W, Meng D, Zhang L, Feng X, Zhang D. A generalized iterated shrinkage algorithm for non-convex sparse coding. In Proc. the 14th IEEE International Conference on Computer Vision, Dec. 2013, pp.217-224.
[26] Boyd S, Parikh N, Chu E, Peleato B, Eckstein J. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trendsr in Machine Learning, 2011, 3(1):1-122.
[27] Chan S H, Wang X, Elgendy O A. Plug-and-play ADMM for image restoration:Fixed-point convergence and applications. IEEE Transactions on Computational Imaging, 2017, 3(1):84-98.
[28] Zhang K, Zuo W, Gu S, Zhang L. Learning deep CNN denoiser prior for image restoration. In Proc. the 30th IEEE Conference on Computer Vision and Pattern Recognition, Jul. 2017, pp.2808-2817.
[29] Schaefer G, Stich M. UCID:An uncompressed color image database. In Proc. the 2004 SPIE Storage and Retrieval Methods and Applications for Multimedia, Dec. 2003, pp.472-480.
[30] Zoran D, Weiss Y. From learning models of natural image patches to whole image restoration. In Proc. the 13th IEEE International Conference on Computer Vision, Nov. 2011, pp.479-486.
[31] Perrone D, Favaro P. Total variation blind deconvolution:The devil is in the details. In Proc. the 27th IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2014, pp.2909-2916.
[32] Lai W S, Huang J B, Hu Z, Ahuja N, Yang M H. A comparative study for single image blind deblurring. In Proc. the 29th IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2016, pp.1701-1709.
[1] Yang Liu, Ruili He, Xiaoqian Lv, Wei Wang, Xin Sun, Shengping Zhang. Is It Easy to Recognize Baby's Age and Gender? [J]. Journal of Computer Science and Technology, 2021, 36(3): 508-519.
[2] Dun Liang, Yuan-Chen Guo, Shao-Kui Zhang, Tai-Jiang Mu, Xiaolei Huang. Lane Detection: A Survey with New Results [J]. Journal of Computer Science and Technology, 2020, 35(3): 493-505.
[3] Bei-Ji Zou, Yun-Di Guo, Qi He, Ping-Bo Ouyang, Ke Liu, Zai-Liang Chen. 3D Filtering by Block Matching and Convolutional Neural Network for Image Denoising [J]. , 2018, 33(4): 838-848.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] Liu Mingye; Hong Enyu;. Some Covering Problems and Their Solutions in Automatic Logic Synthesis Systems[J]. , 1986, 1(2): 83 -92 .
[2] Chen Shihua;. On the Structure of (Weak) Inverses of an (Weakly) Invertible Finite Automaton[J]. , 1986, 1(3): 92 -100 .
[3] Gao Qingshi; Zhang Xiang; Yang Shufan; Chen Shuqing;. Vector Computer 757[J]. , 1986, 1(3): 1 -14 .
[4] Chen Zhaoxiong; Gao Qingshi;. A Substitution Based Model for the Implementation of PROLOG——The Design and Implementation of LPROLOG[J]. , 1986, 1(4): 17 -26 .
[5] Huang Heyan;. A Parallel Implementation Model of HPARLOG[J]. , 1986, 1(4): 27 -38 .
[6] Min Yinghua; Han Zhide;. A Built-in Test Pattern Generator[J]. , 1986, 1(4): 62 -74 .
[7] Tang Tonggao; Zhao Zhaokeng;. Stack Method in Program Semantics[J]. , 1987, 2(1): 51 -63 .
[8] Min Yinghua;. Easy Test Generation PLAs[J]. , 1987, 2(1): 72 -80 .
[9] Zhu Hong;. Some Mathematical Properties of the Functional Programming Language FP[J]. , 1987, 2(3): 202 -216 .
[10] Li Minghui;. CAD System of Microprogrammed Digital Systems[J]. , 1987, 2(3): 226 -235 .

ISSN 1000-9000(Print)

         1860-4749(Online)
CN 11-2296/TP

Home
Editorial Board
Author Guidelines
Subscription
Journal of Computer Science and Technology
Institute of Computing Technology, Chinese Academy of Sciences
P.O. Box 2704, Beijing 100190 P.R. China
Tel.:86-10-62610746
E-mail: jcst@ict.ac.cn
 
  Copyright ©2015 JCST, All Rights Reserved