›› 2012, Vol. 27 ›› Issue (6): 1119-1128.doi: 10.1007/s11390-012-1290-4

Special Issue: Artificial Intelligence and Pattern Recognition

• Special Section on Computational Visual Media • Previous Articles     Next Articles

Affective Image Colorization

Xiao-Hui Wang1 (王晓慧), Jia Jia1 (贾珈), Han-Yu Liao2 (廖捍宇), and Lian-Hong Cai1 (蔡莲红)   

  1. 1. Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China;
    2. Academy of Art & Design, Tsinghua University, Beijing 100084, China
  • Received:2012-09-05 Revised:2012-09-17 Online:2012-11-05 Published:2012-11-05
  • Supported by:

    This work is supported by the National Basic Research 973 Program of China under Grant No. 2011CB302201, the National Natural Science Foundation of China under Grant Nos. 61003094, 60931160443. This work is also funded by Tsinghua National Laboratory for Information Science and Technology (TNList) Cross-Discipline Foundation of China, and supported by the Innovation Fund of Tsinghua-Tencent Joint Laboratory of China.

Colorization of gray-scale images has attracted many attentions for a long time. An important role of image color is the conveyer of emotions (through color themes). The colorization with an undesired color theme is less useful, even it is semantically correct. However this has been rarely considered. Automatic colorization respecting both the semantics and the emotions is undoubtedly a challenge. In this paper, we propose a complete system for affective image colorization. We only need the user to assist object segmentation along with text labels and an affective word. First, the text labels along with other object characters are jointly used to filter the internet images to give each object a set of semantically correct reference images. Second, we select a set of color themes according to the affective word based on art theories. With these themes, a generic algorithm is used to select the best reference for each object, balancing various requirements. Finally, we propose a hybrid texture synthesis approach for colorization. To the best of our knowledge, it is the first system which is able to efficiently colorize a gray-scale image semantically by an emotionally controllable fashion. Our experiments show the effectiveness of our system, especially the benefit compared with the previous Markov random field (MRF) based method.

[1] Levin A, Lischinski D, Weiss Y. Colorization using optimiza-tion. ACM Transactions on Graphics, 2004, 23(3): 689-694.

[2] Welsh T, Ashikhmin M, Mueller K. Transferring color togreyscale images. ACM Transactions on Graphics, 2002,21(3): 277-280.

[3] Ironi R, Cohen-Or D, Lischinski D. Colorization by exam-ple. In Proc. the 16th Eurographics Workshop on RenderingTechniques, June 29-July 1, 2005, pp.201-210.

[4] Charpiat G, Hofmann M, Schölkopf B. Automatic image col-orization via multimodal predictions. In Proc. the 10thECCV, October 2008, pp.126-139.

[5] Tai Y W, Jia J Y, Tang C K. Local color transfer via proba-bilistic segmentation by expectation-maximization. In Proc.CVPR2005, June 2005, Vol.1, pp.747-754.

[6] Chia A Y S, Zhuo S J, Gupta R K, Tai Y W, Cho S Y, TanP, Lin S. Semantic colorization with internet images. ACMTransactions on Graphics, 2011, 30(6), Article No.156.

[7] Arnheim R. Art and Visual Perception: A Psychology of theCreative Eye. University of California Press, 1954.

[8] Kobayashi S. Color Image Scale. Kosdansha International,1992.

[9] Kobayashi S. Art of Color Combinations. Kosdansha Inter-national, 1995.

[10] Wang X H, Jia J, Liao H Y, Cai L H. Image colorizationwith an affective word. In Proc. Computational Visual Me-dia Conference 2012, November 2012, pp.51-58.

[11] Xu K, Li Y, Ju T, Hu S M, Liu T Q. Efficient affinity-based edit propagation using K-D tree. ACM Transactionson Graphics, 2009, 28(5), Article No.118.

[12] Li Y, Ju T, Hu S M. Instant propagation of sparse edits onimages and videos. Computer Graphics Forum, 2010, 29(7):2049-2054.

[13] Huang Y C, Tung Y S, Chen J C, Wang S W, Wu J L. Anadaptive edge detection based colorization algorithm and itsapplications. In Proc. the 13th MULTIMEDIA, November2005, pp.351-354.

[14] Huang H, Zang Y, Li C F. Example-based painting guided bycolor features. The Visual Computer, 2010, 26(6/8): 933-942.

[15] Xiao X Z, Ma L Z. Gradient-preserving color transfer. Com-puter Graphics Forum, 2009, 28(6/8): 1879-1886.

[16] Reinhard E, Adhikhmin M, Gooch B, Shirley P. Color transferbetween images. IEEE Computer Graphics and Applications,2001, 21(5): 34-41.

[17] Chen T, Tan P, Ma L Q, Cheng M M, Shamir A, Hu S M.PoseShop: Human image database construction and person-alized content synthesis. IEEE Transactions on Visualizationand Computer Graphics, 2012, http://doi. ieeecomputersoci-ety.org/10.1109/TVCG.2012.148, Sept. 2012.

[18] Huang H, Zhang L, Zhang H C. Arcimboldo-like collage us-ing internet images. ACM Transactions on Graphics, 2011,30(6), Article No.155.

[19] Liu H, Zhang L, Huang H. Web-image driven best views of3D shapes. The Visual Computer, 2012, 28(3): 279-287.

[20] Chen T, Cheng M M, Tan P, Shamir A, Hu S M.Sketch2Photo: Internet image montage. ACM Transactionson Graphics, 2009, 28(5), Article No.124.

[21] Csurka G, Skaff S, Marchesotti L, Saunders C. Building look& feel concept models from color combinations: With applica-tions in image dessification, retrieval, and color transfer. TheVisual Computer, 2011, 27(12): 1039-1053.

[22] Cohen-Or D, Sorkine O, Gal R, Leyvand T, Xu Y Q. Colorharmonization. ACM Transactions on Graphics, 2006, 25(3):624-630.

[23] O'Donovan P, Agarwala A, Hertzmann A. Color compatibilityfrom large datasets. ACM Transactions on Graphics, 2011,30(4), Article No.63.

[24] Boykov Y, Funka-Lea G. Graph cuts and efficient N-D im-age segmentation. International Journal of Computer Vision,2006, 70(2): 109-131.

[25] Rother C, Kolmogorov V, Blake A. "GrabCut": Interactiveforeground extraction using iterated graph cuts. ACM Trans-actions on Graphics, 2004, 23(3): 309-314.

[26] Cheng M M, Zhang F L, Mitra N J, Huang X L, Hu S M.Repfinder: Finding approximately repeated scene elementsfor image editing. ACM Transactions on Graphics, 2010,29(4), Article, No. 83.

[27] Cheng M M, Zhang G X, Mitra N J, Huang X L, Hu SM. Global contrast based salient region detection. In Proc.CVPR2011, June 2011, pp.409-416.

[28] Belongie S, Malik J, Puzicha J. Shape matching and objectrecognition using shape contexts. IEEE Transactions on Pat-tern Analysis and Machine Intelligence, 2002, 24(4): 509-522.

[29] Wang X H, Jia J, Cai L H. Affective image adjustment witha single word. To appear in The Visual Computer.

[30] Dong Z D, Dong Q. HowNet and the Computation of Mean-ing. World Scientific, 2006.

[31] Luxburg U. A tutorial on spectral clustering. Statistics andComputing, 2007, 17(4): 395-416.

[32] Barnes C, Shechtman E, Finkelstein A, Goldman D B. Patch-match: A randomized correspondence algorithm for struc-tural image editing. ACM Transactions on Graphics, 2009,28(3), Article No.24.

[33] Chen J T, Wang B. Solid texture synthesis using position his-togram matching. In Proc. the 11th CAD/Graphics, August2009, pp.150-153.

[34] Sheikh H R, Bovik A C. Image information and visual qual-ity. IEEE Transactions on Image Processing, 2006, 15(2):430-444.

[35] Zhong F, Qin X Y, Peng Q S. Robust image segmentationagainst complex color distribution. The Visual Computer,2011, 27(6-8): 707-716.

[36] Wu J L, Shen X Y, Liu L G. Interactive two-scale color-to-gray. The Visual Computer, 2012, 28(6-8): 723-731.
No related articles found!
Full text



[1] Li Renwei;. Soundness and Completeness of Kung s Reasoning Procedure[J]. , 1988, 3(1): 7 -15 .
[2] Klaus Buchenrieder;. Standard-Cell Placement from Functional Descriptions[J]. , 1991, 6(1): 37 -46 .
[3] Wu Xindong;. Inductive Learning[J]. , 1993, 8(2): 22 -36 .
[4] Qin Kaihuai; Fan Gang; Sun Cai;. Extrapolating Acceleration Algorithms for Finding B-Spline Intersections Using Recursive Subdivision Techniques[J]. , 1994, 9(1): 70 -85 .
[5] wang Xuejun; Shi Chunyi;. A Multiagent Dynamic interaction Testbed:Theoretic Framework, System Architecture and Experimentation[J]. , 1997, 12(2): 121 -132 .
[6] Dong Yunmei;. An Interactive Learning Algorithm for Acquisition of Concepts Represented as CFL[J]. , 1998, 13(1): 1 -8 .
[7] LU Sanglu; ZHOU Xiaoboand; XIE Li;. A Model for Dynamic Adaptive Coscheduling[J]. , 1999, 14(3): 267 -275 .
[8] Xu-Bin Deng, and Yang-Yong Zhu. L-tree Match: A New Data Extraction Model and Algorithm for Huge Text Stream with Noises[J]. , 2005, 20(6): 763 -773 .
[9] Byounghyun Yoo and Soonhung Han. Representation of Urban Buildings Using Modified Relief Mapping[J]. , 2006, 21(2): 204 -208 .
[10] Xiao-Qing Zheng, Hua-Jun Chen, Zhao-Hui Wu, and Yu-Xin Mao. Dynamic Query Optimization Approach for Semantic Database Grid[J]. , 2006, 21(4): 597 -608 .

ISSN 1000-9000(Print)

CN 11-2296/TP

Editorial Board
Author Guidelines
Journal of Computer Science and Technology
Institute of Computing Technology, Chinese Academy of Sciences
P.O. Box 2704, Beijing 100190 P.R. China
E-mail: jcst@ict.ac.cn
  Copyright ©2015 JCST, All Rights Reserved