We use cookies to improve your experience with our site.

Indexed in:

SCIE, EI, Scopus, INSPEC, DBLP, CSCD, etc.

Submission System
(Author / Reviewer / Editor)
Advanced Search
Volume 32 Issue 3
May  2017
Turn off MathJax
Article Contents
Abstract
References
Related Articles
Wei Zhang, Chao-Wei Fang, Guan-Bin Li. Automatic Colorization with Improved Spatial Coherence and Boundary Localization[J]. Journal of Computer Science and Technology, 2017, 32(3): 494-506. DOI: 10.1007/s11390-017-1739-6
Citation: Wei Zhang, Chao-Wei Fang, Guan-Bin Li. Automatic Colorization with Improved Spatial Coherence and Boundary Localization[J]. Journal of Computer Science and Technology, 2017, 32(3): 494-506. DOI: 10.1007/s11390-017-1739-6
shu

Automatic Colorization with Improved Spatial Coherence and Boundary Localization

  • 1. Department of Computer Science, The University of Hong Kong, Hong Kong, China;
    2. School of Data and Computer Science, Sun Yat-sen University, Guangzhou 510006, China

Funds: This work was partially supported by Hong Kong Research Grants Council under General Research Funds (HKU17209714).
More Information
  • Author Bio:

    Wei Zhang is a Ph.D. candidate at the Department of Computer Science, The University of Hong Kong, Hong Kong. He received his B.E. degree in automation from Chongqing University, Chongqing, in 2010, and M.S. degree in pattern recognition and artificial intelligence from Huazhong University of Science and Technology, Wuhan, in 2013. His research interest covers image colorization and boundary detection.

  • Corresponding author:

    Guan-Bin Li E-mail: liguanbin@mail.sysu.edu.cn

  • Received Date: December 24, 2016
  • Revised Date: February 25, 2017
  • Published Date: May 04, 2017
    Abstract
  • Grayscale image colorization is an important computer graphics problem with a variety of applications. Recent fully automatic colorization methods have made impressive progress by formulating image colorization as a pixel-wise prediction task and utilizing deep convolutional neural networks. Though tremendous improvements have been made, the result of automatic colorization is still far from perfect. Specifically, there still exist common pitfalls in maintaining color consistency in homogeneous regions as well as precisely distinguishing colors near region boundaries. To tackle these problems, we propose a novel fully automatic colorization pipeline which involves a boundary-guided CRF (conditional random field) and a CNN-based color transform as post-processing steps. In addition, as there usually exist multiple plausible colorization proposals for a single image, automatic evaluation for different colorization methods remains a challenging task. We further introduce two novel automatic evaluation schemes to efficiently assess colorization quality in terms of spatial coherence and localization. Comprehensive experiments demonstrate great quality improvement in results of our proposed colorization method under multiple evaluation metrics.
    • FullText(HTML)
    • References (35)
  • [1]
    Levin A, Lischinski D, Weiss Y. Colorization using optimization. ACM Transactions on Graphics (TOG), 2004, 23(3): 689-694.
    [2]
    Huang Y C, Tung Y S, Chen J C, Wang S W, Wu J L. An adaptive edge detection based colorization algorithm and its applications. In Proc. the 13th Annual ACM International Conference on Multimedia, Nov. 2005, pp.351-354.
    [3]
    Luan Q, Wen F, Cohen-Or D, Liang L, Xu Y Q, Shum H Y. Natural image colorization. In Proc. the 18th Eurographics Conference on Rendering Techniques, Jun. 2007, pp.309-320.
    [4]
    Qu Y,Wong T T, Heng P A. Manga colorization. ACM Transactions on Graphics (TOG), 2006, 25(3): 1214-1220.
    [5]
    Zhao H L, Nie G Z, Li X J, Jin X G, Pan Z G. Structureaware nonlocal optimization framework for image colorization. Journal of Computer Science and Technology, 2015, 30(3): 478-488.
    [6]
    Sheng B, Sun H, Magnor M, Li P. Video colorization using parallel optimization in feature space. IEEE Transactions on Circuits and Systems for Video Technology, 2014, 24(3): 407-417.
    [7]
    Welsh T, Ashikhmin M, Mueller K. Transferring color to greyscale images. ACM Transactions on Graphics (TOG), 2002, 21(3): 277-280.
    [8]
    Irony R, Cohen-Or D, Lischinski D. Colorization by example. In Proc. Eurographics Symp. Rendering Techqiques, June 29-July 1, 2005, pp.201-210.
    [9]
    Charpiat G, Hofmann M, Schölkopf B. Automatic image colorization via multimodal predictions. In Proc. the 10th European Conference on Computer Vision, Oct. 2008, pp.126-139.
    [10]
    Liu X, Wan L, Qu Y, Wong T T, Lin S, Leung C S, Heng P A. Intrinsic colorization. ACM Transactions on Graphics (TOG), 2008, 27(5): 152:1-152:9.
    [11]
    Gupta R K, Chia A Y S, Rajan D, Ng E S et al. Image colorization using similar images. In Proc. the 20th ACM International Conference on Multimedia, Oct.29-Nov.2, 2012, pp.369-378.
    [12]
    Jin S Y, Choi H J, Tai Y W. A randomized algorithm for natural object colorization. Computer Graphics Forum, 2014, 33(2): 205-214.
    [13]
    Chia A Y S, Zhuo S, Gupta R K, Tai Y W, Cho S Y, Tan P, Lin S. Semantic colorization with Internet images. ACM Transactions on Graphics (TOG), 2011, 30(6): 156:1-156:8.
    [14]
    Deshpande A, Rock J, Forsyth D. Learning large-scale automatic image colorization. In Proc. the IEEE International Conference on Computer Vision, Dec. 2015, pp.567-575.
    [15]
    Li X, Zhao H, Nie G, Huang H. Image recoloring using geodesic distance based color harmonization. Computational Visual Media, 2015, 1(2): 143-155.
    [16]
    Cheng Z, Yang Q, Sheng B. Deep colorization. In Proc. the IEEE International Conference on Computer Vision, Dec. 2015, pp.415-423.
    [17]
    Dahl R. Automatic colorization. http://tinyclouds.org/colorize/, Aug. 2016.
    [18]
    Larsson G, Maire M, Shakhnarovich G. Learning representations for automatic colorization. In Proc. European Conference on Computer Vision, Oct. 2016, pp.577-593.
    [19]
    Zhang R, Isola P, Efros A A. Colorful image colorization. In Proc. European Conference on Computer Vision, Oct. 2016, pp.649-666.
    [20]
    Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556, 2014. https://arxiv.org/abs/1409.1556, Aug. 2016.
    [21]
    Hariharan B, Arbeláez P, Girshick R, Malik J. Hypercolumns for object segmentation and fine-grained localization. In Proc. the IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2015, pp.447-456.
    [22]
    Noh H, Hong S, Han B. Learning deconvolution network for semantic segmentation. In Proc. the IEEE International Conference on Computer Vision, Dec. 2015, pp.1520-1528.
    [23]
    Li G, Yu Y. Deep contrast learning for salient object detection. In Proc. the IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2016, pp.478-487.
    [24]
    Li G, Yu Y. Visual saliency based on multiscale deep features. In Proc. the IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2015, pp.5455-5463.
    [25]
    Ren S, He K, Girshick R, Sun J. Faster R-CNN: Towards real-time object detection with region proposal networks. In Proc. Advances in Neural Information Processing Systems, Dec. 2015, pp.91-99.
    [26]
    Xie S, Tu Z. Holistically-nested edge detection. In Proc. the IEEE International Conference on Computer Vision, Dec. 2015, pp.1395-1403.
    [27]
    Iizuka S, Simo-Serra E, Ishikawa H. Let there be color!: Joint end-to-end learning of global and local image priors for automatic image colorization with simultaneous classification. ACM Transactions on Graphics (TOG), 2016, 35(4): 110:1-110:11.
    [28]
    Noh H, Hong S, Han B. Learning deconvolution network for semantic segmentation. In Proc. the IEEE International Conference on Computer Vision, Dec. 2015, pp.1520-1528.
    [29]
    Yu F, Koltun V. Multi-scale context aggregation by dilated convolutions. arXiv:1511.07122, 2015. https://arxiv.org/abs/1511.07122, Aug. 2016.
    [30]
    Boykov Y, Veksler O, Zabih R. Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2001, 23(11): 1222- 1239.
    [31]
    He K, Zhang X, Ren S, Sun J. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proc. the IEEE International Conference on Computer Vision, Dec. 2015, pp.1026-1034.
    [32]
    Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T. Caffe: Convolutional architecture for fast feature embedding. In Proc. the 22nd ACM International Conference on Multimedia, Nov. 2014, pp.675-678.
    [33]
    Felzenszwalb P F, Huttenlocher D P. Efficient graph-based image segmentation. International Journal of Computer Vision, 2004, 59(2): 167-181.
    [34]
    Arbeláez P, Pont-Tuset J, Barron J T, Marques F, Malik J. Multiscale combinatorial grouping. In Proc. the IEEE Conference on Computer Vision and Pattern Recognition, June 2014, pp.328-335.
    [35]
    Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M et al. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 2015, 115(3): 211-252.
    • Relative Articles

  • Related Articles

    [1]Yi-Li Fang, Hai-Long Sun, Peng-Peng Chen, Ting Deng. Improving the Quality of Crowdsourced Image Labeling via Label Similarity[J]. Journal of Computer Science and Technology, 2017, 32(5): 877-889. DOI: 10.1007/s11390-017-1770-7
    [2]Qiaohong Li, Yu-Ming Fang, Jing-Tao Xu. A Novel Spatial Pooling Strategy for Image Quality Assessment[J]. Journal of Computer Science and Technology, 2016, 31(2): 225-234. DOI: 10.1007/s11390-016-1623-9
    [3]Wei-Ming Dong, Guan-Bo Bao, Xiao-Peng Zhang, Jean-Claude Paul. Fast Multi-Operator Image Resizing and Evaluation[J]. Journal of Computer Science and Technology, 2012, 27(1): 121-134. DOI: 10.1007/s11390-012-1211-6
    [4]Shou-Xin Wang, Li Zhang, Shuai Wang, Xiang Qiu. A Cloud-Based Trust Model for Evaluating Quality of Web Services[J]. Journal of Computer Science and Technology, 2010, 25(6): 1130-1142. DOI: 10.1007/s11390-010-1090-7
    [5]LIAO Husheng. An Action Analysis for Combining Partial Evaluation[J]. Journal of Computer Science and Technology, 2000, 15(2): 196-201.
    [6]ZHAN Yongzhao, SONG Snunlin, XIE Li. Demand Priority Protocol Simulation and Evaluation[J]. Journal of Computer Science and Technology, 1999, 14(6): 599-605.
    [7]Chen Yangjun. Graph Traversal and Top-Down Evaluation of Logic Queries[J]. Journal of Computer Science and Technology, 1998, 13(4): 300-316.
    [8]Fang Zhiyi, Ju Jiubin. NONH:A New Cache-Based Coherence Protocol for Linked List Structure DSM System and Its Performance Evaluation[J]. Journal of Computer Science and Technology, 1996, 11(4): 405-415.
    [9]Shi Baile, Zhou Aoying. Bottom-up Evaluation of Datalog with Negation[J]. Journal of Computer Science and Technology, 1994, 9(3): 229-244.
    [10]Wang Shunqian, Ye Daxing. On Parallel Evaluation of Ordered Attribute Grammars[J]. Journal of Computer Science and Technology, 1991, 6(4): 347-354.
    • Supplements (0)

Catalog

    Get Citation
    PDF
    XML
    Article views (31) PDF downloads (1062) Cited by()
    Related

    Copyright ©2025 JCST, All Rights Reserved     京ICP备05002829号-1

    Institute of Computing Technology, Chinese Academy of Sciences
    P.O. Box 2704, Beijing 100190, P.R. China

    Tel.: 86-10-62610746 E-mail: jcst@ict.ac.cn

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return