计算机科学技术学报 ›› 2019,Vol. 34 ›› Issue (3): 609-621.doi: 10.1007/s11390-019-1930-z

所属专题: Artificial Intelligence and Pattern Recognition Computer Graphics and Multimedia

• Artificial Intelligence and Pattern Recognition • 上一篇    下一篇

基于灵活稀疏结构控制和自适应优化算法的模糊图像盲复原

Ri-Sheng Liu, Member, ACM, IEEE, Cai-Sheng Mao, Zhi-Hui Wang, Hao-Jie Li*, Member, ACM, IEEE   

  1. International School of Information Science and Engineering, Dalian University of Technology, Dalian 116620, China;Key Laboratory for Ubiquitous Network and Service Software of Liaoning Province, Dalian 116620, China
  • 收稿日期:2018-10-19 修回日期:2019-03-26 出版日期:2019-05-05 发布日期:2019-05-06
  • 通讯作者: Hao-Jie Li E-mail:hjli@dlut.edu.cn
  • 作者简介:Ri-Sheng Liu is currently an associate professor in the International School of Information Science and Engineering, Dalian University of Technology, Dalian. He received his B.S. and Ph.D. degrees both in mathematics from the Dalian University of Technology, Dalian, in 2007 and 2012 respectively. He was a visiting scholar in the Robotic Institute of Carnegie Mellon University, Pittsburgh, from 2010 to 2012. He served as a Hong Kong scholar research fellow at the Hong Kong Polytechnic University, Hong Kong, from 2016 to 2017. He is a member of ACM and IEEE. His research interests include machine learning, optimization, computer vision and multimedia.
  • 基金资助:
    This work was partially supported by the National Natural Science Foundation of China under Grant Nos. 61672125 and 61772108.

Blind Image Deblurring via Adaptive Optimization with Flexible Sparse Structure Control

Ri-Sheng Liu, Member, ACM, IEEE, Cai-Sheng Mao, Zhi-Hui Wang, Hao-Jie Li*, Member, ACM, IEEE   

  1. International School of Information Science and Engineering, Dalian University of Technology, Dalian 116620, China;Key Laboratory for Ubiquitous Network and Service Software of Liaoning Province, Dalian 116620, China
  • Received:2018-10-19 Revised:2019-03-26 Online:2019-05-05 Published:2019-05-06
  • Contact: Hao-Jie Li E-mail:hjli@dlut.edu.cn
  • About author:Ri-Sheng Liu is currently an associate professor in the International School of Information Science and Engineering, Dalian University of Technology, Dalian. He received his B.S. and Ph.D. degrees both in mathematics from the Dalian University of Technology, Dalian, in 2007 and 2012 respectively. He was a visiting scholar in the Robotic Institute of Carnegie Mellon University, Pittsburgh, from 2010 to 2012. He served as a Hong Kong scholar research fellow at the Hong Kong Polytechnic University, Hong Kong, from 2016 to 2017. He is a member of ACM and IEEE. His research interests include machine learning, optimization, computer vision and multimedia.
  • Supported by:
    This work was partially supported by the National Natural Science Foundation of China under Grant Nos. 61672125 and 61772108.

模糊图像的盲复原是一个长期存在的病态逆问题。为了从模糊观测中恢复出清晰图像,目前已有的工作基于最大后验估计设计了很多针对清晰图像的先验以缩小解空间,而过于复杂的非凸先验会造成最终模型优化困难。同时,在现实场景下,由于未知的图像分布,复杂的模糊过程以及不均匀的图像噪声,很难显式地设计出一个固定先验适应于所有情况。因此我们采用了自适应优化的思想,在优化过程中对图像稀疏结构进行控制。在本篇文章中,我们提出了不带有任何先验项的轻量级模型,结合投影梯度法在优化过程中加入了图像稀疏结构控制模块。此外,我们训练了一组卷积神经网络从不同噪声下的训练数据中学习清晰图像的稀疏结构。相比之前以lp范数控制图像稀疏结构的启发式过程,避免了人工调参的繁琐。实验证明基于卷积神经网络的稀疏结构控制结果与lp范数相近,但对图像噪声更为鲁棒。在合成数据集以及真实图像上的实验结果表明,我们提出的方法可以有效应用到不同场景下,且相比目前较新方法有明显优势。

关键词: 图像盲去模糊, 卷积神经网络, 非凸优化, 稀疏结构控制

Abstract: Blind image deblurring is a long-standing ill-posed inverse problem which aims to recover a latent sharp image given only a blurry observation. So far, existing studies have designed many effective priors w.r.t. the latent image within the maximum a posteriori (MAP) framework in order to narrow down the solution space. These non-convex priors are always integrated into the final deblurring model, which makes the optimization challenging. However, due to unknown image distribution, complex kernel structure and non-uniform noises in real-world scenarios, it is indeed challenging to explicitly design a fixed prior for all cases. Thus we adopt the idea of adaptive optimization and propose the sparse structure control (SSC) for the latent image during the optimization process. In this paper, we only formulate the necessary optimization constraints in a lightweight MAP model with no priors. Then we develop an inexact projected gradient scheme to incorporate flexible SSC in MAP inference. Besides lp-norm based SSC in our previous work, we also train a group of denoising convolutional neural networks (CNNs) to learn the sparse image structure automatically from the training data under different noise levels, and we show that CNNs-based SSC can achieve similar results compared with lp-norm but are more robust to noise. Extensive experiments demonstrate that the proposed adaptive optimization scheme with two types of SSC achieves the state-of-the-art results on both synthetic data and real-world images.

Key words: blind image deblurring, convolutional neural network (CNN), non-convex optimization, sparse structure control (SSC)

[1] Pan J, Sun D, Pfister H, Yang M H. Blind image deblurring using dark channel prior. In Proc. the 29th IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2016, pp.1628-1636.
[2] Chan T F, Wong C K. Total variation blind deconvolution. IEEE Transactions on Image Processing, 1998, 7(3):370- 375.
[3] Krishnan D, Fergus R. Fast image deconvolution using hyper-Laplacian priors. In Proc. the 23rd Annual Conference on Neural Information Processing Systems, Dec. 2009, pp.1033-1041.
[4] Levin A, Weiss Y, Durand F, Freeman W T. Efficient marginal likelihood optimization in blind deconvolution. In Proc. the 24th IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2011, pp.2657-2664.
[5] Sun L, Cho S, Wang J, Hays J. Edge-based blur kernel estimation using patch priors. In Proc. the 3rd IEEE International Conference on Computational Photography, Apr. 2013, Article No. 8.
[6] Pan J, Hu Z, Su Z, Yang M H. Deblurring text images via L0-regularized intensity and gradient prior. In Proc. the 27th IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2014, pp.2901-2908.
[7] Perrone D, Diethelm R, Favaro P. Blind deconvolution via lower-bounded logarithmic image priors. In Proc. the 10th International Workshop on Energy Minimization Methods in Computer Vision and Pattern Recognition, Jan. 2015, pp.112-125.
[8] Cho S, Lee S. Fast motion deblurring. ACM Transactions on Graphics, 2009, 28(5):Article No. 145.
[9] Krishnan D, Tay T, Fergus R. Blind deconvolution using a normalized sparsity measure. In Proc. the 24th IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2011, pp.233-240.
[10] Zuo W, Ren D, Zhang D, Gu S, Zhang L. Learning iterationwise generalized shrinkage-thresholding operators for blind deconvolution. IEEE Transactions on Image Processing, 2016, 25(4):1751-1764.
[11] Hu Z, Cho S, Wang J, Yang, M H. Deblurring low-light images with light streaks. In Proc. the 27th IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2014, pp.3382-3389.
[12] Mao C, Liu R, Li H. Adaptive optimization with nested prior navigation for blind image deblurring. In Proc. the 4th IEEE International Conference on Multimedia Big Data, Sept. 2018, Article No. 37.
[13] Xu L, Jia J. Two-phase kernel estimation for robust motion deblurring. In Proc. the 11th European Conference on Computer Vision, Sept. 2010, pp.157-170.
[14] Fergus R, Singh B, Hertzmann A, Roweis S T, Freeman W T. Removing camera shake from a single photograph. ACM Transactions on Graphics, 2006, 25(3):787-794.
[15] Levin A, Weiss Y, Durand F, Freeman W T. Understanding and evaluating blind deconvolution algorithms. In Proc. the 22nd IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2009, pp.1964-1971.
[16] Wipf D, Zhang H. Revisiting Bayesian blind deconvolution. The Journal of Machine Learning Research, 2014, 15(1):3595-3634.
[17] Harmeling S, Sra S, Hirsch M, Schölkopf B. Multiframe blind deconvolution, super-resolution, and saturation correction via incremental EM. In Proc. the 17th IEEE International Conference on Image Processing, Sept. 2010, pp.3313-3316.
[18] Gong D, Tan M, Zhang Y, van den Hengel A, Shi Q. Blind image deconvolution by automatic gradient activation. In Proc. the 29th IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2016, pp.1827-1836.
[19] Zhang K, Zuo W, Chen Y, Meng D, Zhang L. Beyond a Gaussian denoiser:Residual learning of deep CNN for image denoising. IEEE Transactions on Image Processing, 2017, 26(7):3142-3155.
[20] Ng M K, Chan R H, Tang W C. A fast algorithm for deblurring models with Neumann boundary conditions. SIAM Journal on Scientific Computing, 1999, 21(3):851-866.
[21] Donatelli M, Estatico C, Martinelli A, Serra-Capizzano S. Improved image deblurring with anti-reflective boundary conditions and re-blurring. Inverse Problems, 2006, 22(6):2035-2053.
[22] Liu R, Jia J. Reducing boundary artifacts in image deconvolution. In Proc. the 15th IEEE International Conference on Image Processing, Oct. 2008, pp.505-508.
[23] Reeves S J. Fast image restoration without boundary artifacts. IEEE Transactions on Image Processing, 2005, 14(10):1448-1453.
[24] Kruse J, Rother C, Schmidt U. Learning to push the limits of efficient FFT-based image deconvolution. In Proc. the 16th IEEE International Conference on Computer Vision, Oct. 2017, pp.4586-4594.
[25] Zuo W, Meng D, Zhang L, Feng X, Zhang D. A generalized iterated shrinkage algorithm for non-convex sparse coding. In Proc. the 14th IEEE International Conference on Computer Vision, Dec. 2013, pp.217-224.
[26] Boyd S, Parikh N, Chu E, Peleato B, Eckstein J. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trendsr in Machine Learning, 2011, 3(1):1-122.
[27] Chan S H, Wang X, Elgendy O A. Plug-and-play ADMM for image restoration:Fixed-point convergence and applications. IEEE Transactions on Computational Imaging, 2017, 3(1):84-98.
[28] Zhang K, Zuo W, Gu S, Zhang L. Learning deep CNN denoiser prior for image restoration. In Proc. the 30th IEEE Conference on Computer Vision and Pattern Recognition, Jul. 2017, pp.2808-2817.
[29] Schaefer G, Stich M. UCID:An uncompressed color image database. In Proc. the 2004 SPIE Storage and Retrieval Methods and Applications for Multimedia, Dec. 2003, pp.472-480.
[30] Zoran D, Weiss Y. From learning models of natural image patches to whole image restoration. In Proc. the 13th IEEE International Conference on Computer Vision, Nov. 2011, pp.479-486.
[31] Perrone D, Favaro P. Total variation blind deconvolution:The devil is in the details. In Proc. the 27th IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2014, pp.2909-2916.
[32] Lai W S, Huang J B, Hu Z, Ahuja N, Yang M H. A comparative study for single image blind deblurring. In Proc. the 29th IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2016, pp.1701-1709.
[1] 张鑫, 陆思源, 王水花, 余翔, 王甦菁, 姚仑, 潘毅, 张煜东. 通过新型深度学习架构诊断COVID-19肺炎[J]. 计算机科学技术学报, 2022, 37(2): 330-343.
[2] Shao-Jie Qiao, Guo-Ping Yang, Nan Han, Hao Chen, Fa-Liang Huang, Kun Yue, Yu-Gen Yi, Chang-An Yuan. 基数估计器:利用垂直扫描卷积神经网络处理SQL[J]. 计算机科学技术学报, 2021, 36(4): 762-777.
[3] Yang Liu, Ruili He, Xiaoqian Lv, Wei Wang, Xin Sun, Shengping Zhang. 婴儿的年龄和性别容易被识别吗?[J]. 计算机科学技术学报, 2021, 36(3): 508-519.
[4] Zhang-Jin Huang, Xiang-Xiang He, Fang-Jun Wang, Qing Shen. 基于卷积神经网络的实时多阶段斑马鱼头部姿态估计框架[J]. 计算机科学技术学报, 2021, 36(2): 434-444.
[5] 梁盾, 郭元晨, 张少魁, 穆太江, 黄晓蕾. 车道检测-新结果和调查研究[J]. 计算机科学技术学报, 2020, 35(3): 493-505.
[6] Jin-Gong Jia, Yuan-Feng Zhou, Xing-Wei Hao, Feng Li, Christian Desrosiers, Cai-Ming Zhang. 双流时间卷积神经网络用于基于骨架的人体动作识别[J]. 计算机科学技术学报, 2020, 35(3): 538-550.
[7] Rui-Song Zhang, Wei-Ze Quan, Lu-Bin Fan, Li-Ming Hu, Dong-Ming Yan. 基于通道和像素相关性的计算机生成图像与自然图像鉴别[J]. 计算机科学技术学报, 2020, 35(3): 592-602.
[8] Robail Yasrab. SRNET:用于解析奇点的基于浅跳跃连接的卷积神经网络[J]. 计算机科学技术学报, 2019, 34(4): 924-938.
[9] Han Liu, Hang Du, Dan Zeng, Qi Tian. 基于超像素分类和语义分割的云检测算法[J]. 计算机科学技术学报, 2019, 34(3): 622-633.
[10] Dong-Di Zhao, Fan Li, Kashif Sharif, Guang-Min Xia, Yu Wang. 深度卷积神经网络的空间高效量化[J]. 计算机科学技术学报, 2019, 34(2): 305-317.
[11] Tie-Ke He, Hao Lian, Ze-Min Qin, Zhen-Yu Chen, Bin Luo. 一种用于罚金判定的主题模型[J]. , 2018, 33(4): 756-767.
[12] Bei-Ji Zou, Yun-Di Guo, Qi He, Ping-Bo Ouyang, Ke Liu, Zai-Liang Chen. 基于块匹配三维滤波和卷积神经网络的图像去噪[J]. , 2018, 33(4): 838-848.
[13] Zhi-Feng Xie, Yu-Chen Guo, Shu-Han Zhang, Wen-Jun Zhang, Li-Zhuang Ma. 基于深度卷积网络的多重曝光运动估计[J]. , 2018, 33(3): 487-501.
[14] Nai-Ming Yao, Hui Chen, Qing-Pei Guo, Hong-An Wang. 使用一种基于Depth-Patch的深度神经网络的非正面人脸表情识别[J]. , 2017, 32(6): 1172-1185.
[15] Xiang Bai, Zheng Zhang, Hong-Yang Wang, Wei Shen. 指向性边缘框:基于内法向特征的一般物体检测[J]. , 2017, 32(4): 701-713.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] 刘明业; 洪恩宇;. Some Covering Problems and Their Solutions in Automatic Logic Synthesis Systems[J]. , 1986, 1(2): 83 -92 .
[2] 陈世华;. On the Structure of (Weak) Inverses of an (Weakly) Invertible Finite Automaton[J]. , 1986, 1(3): 92 -100 .
[3] 高庆狮; 张祥; 杨树范; 陈树清;. Vector Computer 757[J]. , 1986, 1(3): 1 -14 .
[4] 陈肇雄; 高庆狮;. A Substitution Based Model for the Implementation of PROLOG——The Design and Implementation of LPROLOG[J]. , 1986, 1(4): 17 -26 .
[5] 黄河燕;. A Parallel Implementation Model of HPARLOG[J]. , 1986, 1(4): 27 -38 .
[6] 闵应骅; 韩智德;. A Built-in Test Pattern Generator[J]. , 1986, 1(4): 62 -74 .
[7] 唐同诰; 招兆铿;. Stack Method in Program Semantics[J]. , 1987, 2(1): 51 -63 .
[8] 闵应骅;. Easy Test Generation PLAs[J]. , 1987, 2(1): 72 -80 .
[9] 朱鸿;. Some Mathematical Properties of the Functional Programming Language FP[J]. , 1987, 2(3): 202 -216 .
[10] 李明慧;. CAD System of Microprogrammed Digital Systems[J]. , 1987, 2(3): 226 -235 .
版权所有 © 《计算机科学技术学报》编辑部
本系统由北京玛格泰克科技发展有限公司设计开发 技术支持:support@magtech.com.cn
总访问量: