›› 2017,Vol. 32 ›› Issue (3): 417-429.doi: 10.1007/s11390-017-1733-z

所属专题: Artificial Intelligence and Pattern Recognition

• Special Section on Selected Paper from NPC 2011 • 上一篇    下一篇

基于细节字典学习的照片表面增强

Zhi-Feng Xie1,2, Shi Tang1, Dong-Jin Huang1,2, You-Dong Ding1,2, Member, CCF, Li-Zhuang Ma2,3, Member, CCF   

  1. 1. Department of Film and Television Engineering, Shanghai University, Shanghai 200072, China;
    2. Shanghai Engineering Research Center of Motion Picture Special Effects, Shanghai 200072, China;
    3. Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
  • 收稿日期:2016-12-09 修回日期:2017-03-09 出版日期:2017-05-05 发布日期:2017-05-05
  • 作者简介:Zhi-Feng Xie received his Ph.D. degree in computer application technology from Shanghai Jiao Tong University, Shanghai, in 2013. He was a research assistant at the Department of Computer Science, City University of Hong Kong, Hong Kong, in 2011. He is now an assistant professor with Shanghai University, Shanghai. His research interests include image/video editing, computer graphics, and digital media technology.
  • 基金资助:

    This work was supported by the National Natural Science Foundation of China under Grant Nos.61303093,61402278,and 61472245,the Innovation Program of the Science and Technology Commission of Shanghai Municipality of China under Grant No.16511101300,the Natural Science Foundation of Shanghai under Grant No.14ZR1415800,and the Gaofeng Film Discipline Grant of Shanghai Municipal Education Commission.

Photographic Appearance Enhancement via Detail-based Dictionary Learning

Zhi-Feng Xie1,2, Shi Tang1, Dong-Jin Huang1,2, You-Dong Ding1,2, Member, CCF, Li-Zhuang Ma2,3, Member, CCF   

  1. 1. Department of Film and Television Engineering, Shanghai University, Shanghai 200072, China;
    2. Shanghai Engineering Research Center of Motion Picture Special Effects, Shanghai 200072, China;
    3. Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
  • Received:2016-12-09 Revised:2017-03-09 Online:2017-05-05 Published:2017-05-05
  • Contact: 10.1007/s11390-017-1733-z
  • About author:Zhi-Feng Xie received his Ph.D. degree in computer application technology from Shanghai Jiao Tong University, Shanghai, in 2013. He was a research assistant at the Department of Computer Science, City University of Hong Kong, Hong Kong, in 2011. He is now an assistant professor with Shanghai University, Shanghai. His research interests include image/video editing, computer graphics, and digital media technology.
  • Supported by:

    This work was supported by the National Natural Science Foundation of China under Grant Nos.61303093,61402278,and 61472245,the Innovation Program of the Science and Technology Commission of Shanghai Municipality of China under Grant No.16511101300,the Natural Science Foundation of Shanghai under Grant No.14ZR1415800,and the Gaofeng Film Discipline Grant of Shanghai Municipal Education Commission.

许多边缘感知的滤波器通过细节分解和增强,能够高效地增强图像的表面。但是,由于一些可视化的表面缺陷,尤其是噪声、晕、不自然的对比度等,这些方法经常不能生成照片级表面的增强效果。关键的原因在于增强过程中对高质量表面的引导与约束不够充分。因此,本文期望从大量高质量图像小块中训练一个细节字典,进一步约束和控制整个表面增强。在这篇论文中,我们提出一种基于学习的照片级表面增强方法,包括两个主要步骤:字典训练和稀疏重建。在训练阶段,我们从很多高质量的照片中抽取大量的细节小块组成训练集,然后通过迭代地优化一个L1-norm能量方程,训练一个过完备的细节字典。在重建阶段,我们利用训练的字典重建增强的细节层,并且形式化一个梯度引导的优化方程来提升小块间的局部连贯性。此外,我们提出了两种评价方式来评测表面增强方法的效果。最终的实验结果验证了我们基于学习的增强方法的有效性。

Abstract: A number of edge-aware filters can efficiently boost the appearance of an image by detail decomposition and enhancement. However, they often fail to produce photographic enhanced appearance due to some visible artifacts, especially noise, halos and unnatural contrast. The essential reason is that the guidance and constraint of high-quality appearance aren't enough sufficient in the process of enhancement. Thus our idea is to train a detail dictionary from a lot of high-quality patches in order to constrain and control the entire appearance enhancement. In this paper, we propose a novel learning-based enhancement method for photographic appearance, which includes two main stages: dictionary training and sparse reconstruction. In the training stage, we construct a training set of detail patches by extracting from some high-quality photos, and then train an overcomplete detail dictionary by iteratively minimizing an L1-norm energy function. In the reconstruction stage, we employ the trained dictionary to reconstruct the boosted detail layer, and further formalize a gradient-guided optimization function to improve the local coherence between patches. Moreover, we propose two evaluation metrics to measure the performance of appearance enhancement. The final experimental results have demonstrated the effectiveness of our learning-based enhancement method.

[1] Paris S, Durand F. A fast approximation of the bilateral filter using a signal processing approach. In Proc. the 9th ECCV, May 2006, pp.568-580.

[2] He K M, Sun J, Tang X O. Guided image filtering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(6): 1397-1409.

[3] Farbman Z, Fattal R, Lischinski D, Szeliski R. Edgepreserving decompositions for multi-scale tone and detail manipulation. ACM Transactions on Graphics, 2008, 27(3): 67:1-67:10.

[4] Paris S, Hasinoff S, Kautz J. Local Laplacian filters: Edgeaware image processing with a Laplacian pyramid. ACM Transactions on Graphics, 2011, 30(4): 68:1-68:12.

[5] Xu L, Lu C W, Xu Y, Jia J Y. Image smoothing via L0 gradient minimization. ACM Transactions on Graphics, 2011, 30(6): 174:1-174:12.

[6] Zhang Q, Shen X Y, Xu L, Jia J Y. Rolling guidance filter. In Proc. the 13th ECCV, Sept. 2014, pp.815-830.

[7] Wright J, Ma Y, Mairal J, Sapiro G, Huang T S, Yan S. Sparse representation for computer vision and pattern recognition. Proceedings of the IEEE, 2010, 98(6): 1031- 1044.

[8] Aharon M, Elad M, Bruckstein A. K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Transactions on Signal Processing, 2006, 54(11): 4311-4322.

[9] Perona P, Malik J. Scale-space and edge detection using anisotropic diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1990, 12: 629-639.

[10] Tomasi C, Manduchi R. Bilateral filtering for gray and color images. In Proc. ICCV, Jan. 1998, pp.839-846.

[11] Durand F, Dorsey J. Fast bilateral filtering for the display of high-dynamic-range images. ACM Transactions on Graphics, 2002, 21(3): 257-266.

[12] Bae S, Paris S, Durand F. Two-scale tone management for photographic look. ACM Transactions on Graphics, 2006, 25(3): 637-645.

[13] Weiss B. Fast median and bilateral filtering. ACM Transactions on Graphics, 2006, 25(3): 519-526.

[14] Chen J W, Paris S, Durand F. Real-time edge-aware image processing with the bilateral grid. ACM Transactions on Graphics, 2007, 26(3): 103:1-103:9.

[15] Porikli F. Constant time O(1) bilateral filtering. In Proc. CVPR, June 2008.

[16] Yang Q X, Tan K H, Ahuja N. Real-time O(1) bilateral filtering. In Proc. CVPR, June 2009, pp.557-564.

[17] Fattal R. Edge-avoiding wavelets and their applications. ACM Transactions on Graphics, 2009, 28(3): 22:1-22:10.

[18] Subr K, Soler C, Durand F. Edge-preserving multiscale image decomposition based on local extrema. ACM Transactions on Graphics, 2009, 28(5): 147:1-147:9.

[19] Gastal E, Oliveira M. Domain transform for edge-aware image and video processing. ACM Transactions on Graphics, 2011, 30(4): 69:1-69:12.

[20] Li X Y, Gu Y, Hu S M, Martin R. Mixed-domain edgeaware image manipulation. IEEE Transactions on Image Processing, 2013, 22(5): 1915-1925.

[21] Wang H, Cao J J, Liu X P, Wang J M, Fan T R, Hu J P. Least-squares images for edge-preserving smoothing. Computational Visual Media, 2015, 1(1): 27-35.

[22] Shao P, Ding S H, Ma L Z, Wu Y S, Wu Y J. Edgepreserving image decomposition via joint weighted least squares. Computational Visual Media, 2015, 1(1): 37-47.

[23] Xu L, Ren J, Yan Q, Liao R J, Jia J Y. Deep edge-aware filters. In Proc. the 32nd ICML, July 2015, pp.1669-1678.

[24] Gonzalez R, Woods R. Digital Image Processing (3rd edition). Upper Saddle River, NJ, USA: Prentice-Hall, 2006.

[25] Stark J A. Adaptive image contrast enhancement using generalizations of histogram equalization. IEEE Transactions on Image Processing, 2000, 9(5): 889-896.

[26] Arici T, Dikbas S, Altunbasak Y. A histogram modification framework and its application for image contrast enhancement. IEEE Transactions on Image Processing, 2009, 18(9): 1921-1935.

[27] Celik T, Tjahjadi T. Contextual and variational contrast enhancement. IEEE Transactions on Image Processing, 2011, 20(12): 3431-3441.

[28] Celik T. Two-dimensional histogram equalization and contrast enhancement. Pattern Recognition, 2012, 45(10): 3810-3824.

[29] Fattal R, Agrawala M, Rusinkiewicz S. Multiscale shape and detail enhancement from multi-light image collections. ACM Transactions on Graphics, 2007, 26(3): 51:1-51:10.

[30] Joshi N, Matusik W, Adelson E, Kriegman D. Personal photo enhancement using example images. ACM Transactions on Graphics, 2010, 29(2): 12:1-12:15.

[31] Kang S B, Kapoor A, Lischinski D. Personalization of image enhancement. In Proc. the 23rd CVPR, June 2010, pp.1799- 1806.

[32] Caicedo J C, Kapoor A, Kang S B. Collaborative personalization of image enhancement. In Proc. the 24th CVPR, June 2011, pp.249-256.

[33] Bychkovsky V, Paris S, Chan E, Durand F. Learning photographic global tonal adjustment with a database of input/output image pairs. In Proc. the 24th CVPR, June 2011, pp.97-104.

[34] Kaufman L, Lischinski D, Werman M. Content-aware automatic photo enhancement. Computer Graphics Forum, 2012, 31(8): 2528-2540.

[35] Lischinski D, Farbman Z, Uyttendaele M, Szeliski R. Interactive local adjustment of tonal values. ACM Transactions on Graphics, 2006, 25(3): 646-653.

[36] Zhang F, Zhang X, Qin X Y, Zhang C M. Enlarging image by constrained least square approach with shape preserving. Journal of Computer Science and Technology, 2015, 30(3): 489-498.

[37] Liu Q, Chen M Y, Zhou D H. Single image haze removal via depth-based contrast stretching transform. Science China: Information Sciences, 2015, 58(1): 1-17.

[38] Zhang Q, Nie Y W, Zhang L, Xiao C X. Underexposed video enhancement via perception-driven progressive fusion. IEEE Transactions on Visualization and Computer Graphics, 2016, 22(6): 1773-1785.

[39] Cheng M M, Hou Q B, Zhang S H, Rosin P. Intelligent visual media processing: When graphics meets vision. Journal of Computer Science and Technology, 2017, 32(1): 110-121.

[40] DeCarlo D, Santella A. Stylization and abstraction of photographs. ACM Transactions on Graphics, 2002, 21(3): 769-776.

[41] Winnemoller H, Olsen S, Gooch B. Real-time video abstraction. ACM Transactions on Graphics, 2006, 25(3): 1221- 1226.

[42] Echevarria J, Wilensky G, Krishnaswamy A, Kim B, Gutierrez D. Computational simulation of alternative photographic processes. Computer Graphics Forum, 2013, 32(4): 7-16.

[43] Son M J, Lee Y J, Kang H, Lee S Y. Art-photographic detail enhancement. Computer Graphics Forum, 2014, 33(2): 391-400.

[44] Aydin T O, Smolic A, Gross M. Automated aesthetic analysis of photographic images. IEEE Transactions on Visualization and Computer Graphics, 2015, 21(1): 31-42.

[45] Mairal J, Sapiro G, Elad M. Learning multiscale sparse representations for image and video restoration. Multiscale Modeling & Simulation, 2008, 7(1): 214-241.

[46] Dong W, Zhang L, Shi G, Li X. Nonlocally centralized sparse representation for image restoration. IEEE Transactions on Image Processing, 2013, 22(4): 1620-1630.

[47] Yang J, Wright J, Huang T S, Ma Y. Image super-resolution via sparse representation. IEEE Transactions on Image Processing, 2010, 19(11): 2861-2873.

[48] Zhang Y, Liu J, Yang W, Guo Z. Image super-resolution based on structure-modulated sparse representation. IEEE Transactions on Image Processing, 2015, 24(9): 2797-2810.

[49] Elad M, Aharon M. Image denoising via sparse and redundant representations over learned dictionaries. IEEE Transactions on Image Processing, 2006, 15(12): 3736-3745.

[50] Zeng X, Bian W, Liu W, Shen J, Tao D. Dictionary pair learning on Grassmann manifolds for image denoising. IEEE Transactions on Image Processing, 2015, 24(11): 4556-4569.
No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] 张钹; 张铃;. Statistical Heuristic Search[J]. , 1987, 2(1): 1 -11 .
[2] 孟力明; 徐晓飞; 常会友; 陈光熙; 胡铭曾; 李生;. A Tree-Structured Database Machine for Large Relational Database Systems[J]. , 1987, 2(4): 265 -275 .
[3] 林琦; 夏培肃;. The Design and Implementation of a Very Fast Experimental Pipelining Computer[J]. , 1988, 3(1): 1 -6 .
[4] 孙成政; 慈云桂;. A New Method for Describing the AND-OR-Parallel Execution of Logic Programs[J]. , 1988, 3(2): 102 -112 .
[5] 张钹; 张恬; 张建伟; 张铃;. Motion Planning for Robots with Topological Dimension Reduction Method[J]. , 1990, 5(1): 1 -16 .
[6] 王鼎兴; 郑纬民; 杜晓黎; 郭毅可;. On the Execution Mechanisms of Parallel Graph Reduction[J]. , 1990, 5(4): 333 -346 .
[7] 周权; 魏道政;. A Complete Critical Path Algorithm for Test Generation of Combinational Circuits[J]. , 1991, 6(1): 74 -82 .
[8] 赵靓海; 刘慎权;. An Environment for Rapid Prototyping of Interactive Systems[J]. , 1991, 6(2): 135 -144 .
[9] 商陆军; 许立辉;. Notes on the Design of an Integrated Object-Oriented DBMS Family[J]. , 1991, 6(4): 389 -394 .
[10] 许建国; 郭玉钗; 林宗楷;. HEPAPS:A PCB Automatic Placement System[J]. , 1992, 7(1): 39 -46 .
版权所有 © 《计算机科学技术学报》编辑部
本系统由北京玛格泰克科技发展有限公司设计开发 技术支持:support@magtech.com.cn
总访问量: