计算机科学技术学报 ›› 2020,Vol. 35 ›› Issue (3): 592-602.doi: 10.1007/s11390-020-0216-9

所属专题: Artificial Intelligence and Pattern Recognition Computer Graphics and Multimedia

• Artificial Intelligence and Pattern Recognition • 上一篇    下一篇

基于通道和像素相关性的计算机生成图像与自然图像鉴别

Rui-Song Zhang1,2, Wei-Ze Quan1,2, Lu-Bin Fan3, Li-Ming Hu4, Dong-Ming Yan1,2,4,*, Senior Member, CCF, Member, ACM, IEEE   

  1. 1 National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China;
    2 School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China;
    3 Alibaba Group, Hangzhou 310023, China;
    4 State Key Laboratory of Hydro-Science and Engineering, Tsinghua University, Beijing 100084, China
  • 收稿日期:2019-12-10 修回日期:2020-01-05 出版日期:2020-05-28 发布日期:2020-05-28
  • 通讯作者: Dong-Ming Yan E-mail:yandongming@gmail.com
  • 作者简介:Rui-Song Zhang is currently pursuing his M.S. degree in the National Laboratory of Pattern Recognition (NLPR), Institute of Automation, Chinese Academy of Sciences, Beijing. He received his B.S. degree in information security from Xidian University, Xi'an, in 2019. His research interests include image processing and image forensics.
  • 基金资助:
    This work was supported by the National Key Research and Development Program of China under Grant No. 2019YFB2204104, the Beijing Natural Science Foundation of China under Grant No. L182059, the National Natural Science Foundation of China under Grant Nos. 61772523, 61620106003, and 61802406, Alibaba Group through Alibaba Innovative Research Program, and the Joint Open Research Fund Program of State Key Laboratory of Hydroscience and Engineering and Tsinghua-Ningxia Yinchuan Joint Institute of Internet of Waters on Digital Water Governance.

Distinguishing Computer-Generated Images from Natural Images Using Channel and Pixel Correlation

Rui-Song Zhang1,2, Wei-Ze Quan1,2, Lu-Bin Fan3, Li-Ming Hu4, Dong-Ming Yan1,2,4,*, Senior Member, CCF, Member, ACM, IEEE        

  1. 1 National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China;
    2 School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China;
    3 Alibaba Group, Hangzhou 310023, China;
    4 State Key Laboratory of Hydro-Science and Engineering, Tsinghua University, Beijing 100084, China
  • Received:2019-12-10 Revised:2020-01-05 Online:2020-05-28 Published:2020-05-28
  • Contact: Dong-Ming Yan E-mail:yandongming@gmail.com
  • About author:Rui-Song Zhang is currently pursuing his M.S. degree in the National Laboratory of Pattern Recognition (NLPR), Institute of Automation, Chinese Academy of Sciences, Beijing. He received his B.S. degree in information security from Xidian University, Xi'an, in 2019. His research interests include image processing and image forensics.
  • Supported by:
    This work was supported by the National Key Research and Development Program of China under Grant No. 2019YFB2204104, the Beijing Natural Science Foundation of China under Grant No. L182059, the National Natural Science Foundation of China under Grant Nos. 61772523, 61620106003, and 61802406, Alibaba Group through Alibaba Innovative Research Program, and the Joint Open Research Fund Program of State Key Laboratory of Hydroscience and Engineering and Tsinghua-Ningxia Yinchuan Joint Institute of Internet of Waters on Digital Water Governance.

近年来随着计算机图形渲染和图像编辑技术的发展,通常不能反映真实场景的计算机生成图像现在可以轻易欺骗人类视觉系统。这导致图像可靠性降低等一系列多媒体安全问题。在本项研究中,我们提出一种基于卷积神经网络(CNN)的模型,通过图像通道、像素相关性来鉴别计算机生成图像与自然图像。本研究利用混合相关性模块提取通道相关性和像素相关性。混合相关性模块中的关键组件是自编码模块,该组件以RGB图像为输入,显式提取颜色通道之间的相关性。其后无池化的卷积层提取像素间的空间相关性。与之前直接使用CNN来解决鉴别问题的方法不同,我们考虑模块的通用性,即新引入的混合相关性模块可以直接与现有的鉴别网络模型结合,增强原网络的鉴别能力。实验结果表明,该方法在鉴别性能上优于现有的鉴别方法。实验同时证明新引入的混合相关性模块可以提高不同CNN架构的鉴别精度,并且带有混合相关性模块的网络在不同压缩率的JPEG压缩和不同数据集上的鉴别性能均更优。
1、目的(Objective):多元化图像处理技术的出现导致图像可靠性降低等一系列多媒体安全问题。针对计算机生成图像和自然图像的鉴别问题,本研究希望提升鉴别渲染图像的准确率、对后处理的鲁棒性和对不同渲染算法的泛化性。从而提升数字媒体的公信力,恢复视觉信息的可靠性。
2、方法(Method):本研究在卷积神经网络前端添加了一个新设计的混合相关性模块。混合相关性模块中的自编码模块提取RGB图像的通道相关性,随后的无池化卷积操作提取像素间的空间相关性。结论是通过比较卷积神经网络有无混合相关性模块的鉴别性能,以及可视化自编码模块卷积核和特征映射得到的。
3、结果(Result & Findings):在新提出网络和三个现有网络的比较实验中,添加三个独立混合相关性模块的网络鉴别准确率平均比标准卷积网络高1.72%;含有自编码模块的网络鉴别准确率平均比没有自编码模块的网络高0.46%。并且带有混合相关性模块的网络在不同压缩率的JPEG压缩和不同数据集上的检测性能均更优。这表明该模块设计在复杂的实际场景有较好的使用前景,但此模块目前仅证明对计算机生成图像鉴别有效。
4、结论(Conclusions):引入RGB图像通道间的相关性和像素间的空间相关性可以提升对计算机生成图像的鉴别准确率。并且两种相关性对后处理的鲁棒性和多种生成方法的泛化性也有一定的提升。这表明两种相关性可以较好刻画计算机生成图像和自然图像在统计特性上的差异。目前针对全局生成图像算法的检测是有效的,可进一步研究其对重上色检测、篡改检测等其他问题的实用性。

关键词: 自然图像, 计算机生成图像, 通道和像素相关, 卷积神经网络

Abstract: With the recent tremendous advances of computer graphics rendering and image editing technologies, computergenerated fake images, which in general do not reflect what happens in the reality, can now easily deceive the inspection of human visual system. In this work, we propose a convolutional neural network (CNN)-based model to distinguish computergenerated (CG) images from natural images (NIs) with channel and pixel correlation. The key component of the proposed CNN architecture is a self-coding module that takes the color images as input to extract the correlation between color channels explicitly. Unlike previous approaches that directly apply CNN to solve this problem, we consider the generality of the network (or subnetwork), i.e., the newly introduced hybrid correlation module can be directly combined with existing CNN models for enhancing the discrimination capacity of original networks. Experimental results demonstrate that the proposed network outperforms state-of-the-art methods in terms of classification performance. We also show that the newly introduced hybrid correlation module can improve the classification accuracy of different CNN architectures.

Key words: natural image, computer-generated image, channel and pixel correlation, convolutional neural network

[1] He P, Jiang X, Sun T, Li H. Computer graphics identification combining convolutional and recurrent neural networks. IEEE Signal Processing Letters, 2018, 25(9):1369-1373.
[2] Ng T T, Chang S F, Hsu J, Xie L, Tsui M P. Physicsmotivated features for distinguishing photographic images and computer graphics. In Proc. the 13th ACM International Conference on Multimedia, November 2005, pp.239-248.
[3] Lyu S, Farid H. How realistic is photorealistic?IEEE Transactions on Signal Processing, 2005, 53(2):845-850.
[4] Chen W, Shi Y Q, Xuan G. Identifying computer graphics using HSV color model and statistical moments of characteristic functions. In Proc. the 2007 IEEE International Conference on Multimedia&Expo, July 2007, pp.1123-1126.
[5] Gallagher A C, Chen T. Image authentication by detecting traces of demosaicing. In Proc. the 2008 IEEE Conference on Computer Vision and Pattern Recognition, June 2008.
[6] Sankar G, Zhao H V, Yang Y H. Feature based classification of computer graphics and real images. In Proc. the IEEE International Conference on Acoustics, Speech and Signal Processing, April 2009, pp.1513-1516.
[7] Peng F, Zhou D l, Long M, Sun X M. Discrimination of natural images and computer generated graphics based on multi-fractal and regression analysis. AEU-International Journal of Electronics and Communications, 2017, 71:72-81.
[8] Rahmouni N, Nozick V, Yamagishi J, Echizen I. Distinguishing computer graphics from natural images using convolution neural networks. In Proc. the 2017 IEEE International Workshop on Information Forensics and Security, December 2017.
[9] Yu I, Kim D, Park J, Hou J, Choi S, Lee H. Identifying photorealistic computer graphics using convolutional neural networks. In Proc. the IEEE International Conference on Image Processing, September 2017, pp.4093-4097.
[10] Quan W, Wang K, Yan D M, Zhang X. Distinguishing between natural and computer-generated images using convolutional neural networks. IEEE Transactions on Information Forensics and Security, 2018, 13(11):2772-2787.
[11] Yao Y, Hu W, Zhang W, Wu T, Shi Y Q. Distinguishing computer-generated graphics from natural images based on sensor pattern noise and deep learning. Sensors, 2018, 18(4):Article No. 1296.
[12] Nguyen H H, Yamagishi J, Echizen I. Capsule-forensics:Using capsule networks to detect forged images and videos. In Proc. the 2019 IEEE International Conference on Acoustics, Speech and Signal Processing, May 2019, pp.2307-2311.
[13] Popescu A C, Farid H. Exposing digital forgeries by detecting traces of resampling. IEEE Transactions on Signal Processing, 2005, 53(2):758-767.
[14] Ianeva T I, de Vries A P, Röhrig H. Detecting cartoons:A case study in automatic video-genre classification. In Proc. the 2003 IEEE International Conference on Multimedia&Expo, July 2003, pp.449-452.
[15] Pan F, Chen J, Huang J. Discriminating between photorealistic computer graphics and natural images using fractal geometry. Science in China Series F:Information Sciences, 2009, 52(2):329-337.
[16] Zhang R, Wang R D, Ng T T. Distinguishing photographic images and photorealistic computer graphics using visual vocabulary on local image edges. In Proc. the 10th International Workshop on Digital Forensics and Watermarking, October 2012, pp.292-305.
[17] Li Z, Ye J, Shi Y Q. Distinguishing computer graphics from photographic images using local binary patterns. In Proc. the 11th International Workshop on Digital Forensics and Watermarking, October 2012, pp.228-241.
[18] Ojala T, Pietikäinen M, Maenpaa T. Multiresolution grayscale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(7):971-987.
[19] LeCun Y, Bengio Y, Hinton G. Deep learning. Nature, 2015, 521(7553):436-444.
[20] Zhang X J, Lu Y F, Zhang S H. Multi-task learning for food identification and analysis with deep convolutional neural networks. Journal of Computer Science and Technology, 2016, 31(3):489-500.
[21] Cheng M M, Hou Q B, Zhang S H, Rosin P L. Intelligent visual media processing:When graphics meets vision. Journal of Computer Science and Technology, 2017, 32(1):110-121.
[22] Quan W, Yan D M, Guo J, Meng W, Zhang X. Maximal Poisson-disk sampling via sampling radius optimization. In Proc. the SIGGRAPH ASIA, December 2016, Article No. 22.
[23] Barnes C, Zhang F L. A survey of the state-of-the-art in patch-based synthesis. Computational Visual Media, 2017, 3(1):3-20.
[24] Hinton G E, Krizhevsky A, Wang S D. Transforming autoencoders. In Proc. the 21st International Conference on Artificial Neural Networks, June 2011, pp.44-51.
[25] Sabour S, Frosst N, Hinton G E. Dynamic routing between capsules. In Proc. the 2017 Annual Conference on Neural Information Processing Systems, December 2017, pp.3856-3866.
[26] Tarianga B D, Senguptab P, Roy A, Chakraborty R S, Naskar R. Classification of computer generated and natural images based on efficient deep convolutional recurrent attention model. In Proc. the IEEE Conference on Computer Vision and Pattern Recognition Workshops, June 2019, pp.146-152.
[27] Yan Y, Ren W, Cao X. Recolored image detection via a deep discriminative model. IEEE Transactions on Information Forensics and Security, 2019, 14(1):5-17.
[28] Bottou L. Large-scale machine learning with stochastic gradient descent. In Proc. the 19th International Conference on Computational Statistics, August 2010, pp.177-186.
[29] Dang-Nguyen D T, Pasquini C, Conotter V, Boato G. RAISE:A raw images dataset for digital image forensics. In Proc. the 6th ACM Multimedia Systems Conference, March 2015, pp.219-224.
[30] Ng T T, Chang S F, Hsu J, Pepeljugoski M. Columbia photographic images and photorealistic computer graphics dataset. Technical Report, Department of Electrical Engineering, Columbia University, 2004. http://www.ee.columbia.edu/ln/dvmm/publications/05/ng cgdataset 05.pdf, Jan. 2019.
[31] Gunturk B K, Altunbasak Y, Mersereau R M. Color plane interpolation using alternating projections. IEEE Transactions on Image Processing, 2002, 11(9):997-1013.
[1] 魏华鹏, 邓盈盈, 唐帆, 潘兴甲, 董未名. 基于卷积神经网络和Transformer的视觉风格迁移的比较研究[J]. 计算机科学技术学报, 2022, 37(3): 601-614.
[2] 张鑫, 陆思源, 王水花, 余翔, 王甦菁, 姚仑, 潘毅, 张煜东. 通过新型深度学习架构诊断COVID-19肺炎[J]. 计算机科学技术学报, 2022, 37(2): 330-343.
[3] Shao-Jie Qiao, Guo-Ping Yang, Nan Han, Hao Chen, Fa-Liang Huang, Kun Yue, Yu-Gen Yi, Chang-An Yuan. 基数估计器:利用垂直扫描卷积神经网络处理SQL[J]. 计算机科学技术学报, 2021, 36(4): 762-777.
[4] Yang Liu, Ruili He, Xiaoqian Lv, Wei Wang, Xin Sun, Shengping Zhang. 婴儿的年龄和性别容易被识别吗?[J]. 计算机科学技术学报, 2021, 36(3): 508-519.
[5] Zhang-Jin Huang, Xiang-Xiang He, Fang-Jun Wang, Qing Shen. 基于卷积神经网络的实时多阶段斑马鱼头部姿态估计框架[J]. 计算机科学技术学报, 2021, 36(2): 434-444.
[6] 梁盾, 郭元晨, 张少魁, 穆太江, 黄晓蕾. 车道检测-新结果和调查研究[J]. 计算机科学技术学报, 2020, 35(3): 493-505.
[7] Jin-Gong Jia, Yuan-Feng Zhou, Xing-Wei Hao, Feng Li, Christian Desrosiers, Cai-Ming Zhang. 双流时间卷积神经网络用于基于骨架的人体动作识别[J]. 计算机科学技术学报, 2020, 35(3): 538-550.
[8] Robail Yasrab. SRNET:用于解析奇点的基于浅跳跃连接的卷积神经网络[J]. 计算机科学技术学报, 2019, 34(4): 924-938.
[9] Ri-Sheng Liu, Cai-Sheng Mao, Zhi-Hui Wang, Hao-Jie Li. 基于灵活稀疏结构控制和自适应优化算法的模糊图像盲复原[J]. 计算机科学技术学报, 2019, 34(3): 609-621.
[10] Han Liu, Hang Du, Dan Zeng, Qi Tian. 基于超像素分类和语义分割的云检测算法[J]. 计算机科学技术学报, 2019, 34(3): 622-633.
[11] Dong-Di Zhao, Fan Li, Kashif Sharif, Guang-Min Xia, Yu Wang. 深度卷积神经网络的空间高效量化[J]. 计算机科学技术学报, 2019, 34(2): 305-317.
[12] Tie-Ke He, Hao Lian, Ze-Min Qin, Zhen-Yu Chen, Bin Luo. 一种用于罚金判定的主题模型[J]. , 2018, 33(4): 756-767.
[13] Bei-Ji Zou, Yun-Di Guo, Qi He, Ping-Bo Ouyang, Ke Liu, Zai-Liang Chen. 基于块匹配三维滤波和卷积神经网络的图像去噪[J]. , 2018, 33(4): 838-848.
[14] Zhi-Feng Xie, Yu-Chen Guo, Shu-Han Zhang, Wen-Jun Zhang, Li-Zhuang Ma. 基于深度卷积网络的多重曝光运动估计[J]. , 2018, 33(3): 487-501.
[15] Nai-Ming Yao, Hui Chen, Qing-Pei Guo, Hong-An Wang. 使用一种基于Depth-Patch的深度神经网络的非正面人脸表情识别[J]. , 2017, 32(6): 1172-1185.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] 周笛;. A Recovery Technique for Distributed Communicating Process Systems[J]. , 1986, 1(2): 34 -43 .
[2] 陈世华;. On the Structure of Finite Automata of Which M Is an(Weak)Inverse with Delay τ[J]. , 1986, 1(2): 54 -59 .
[3] 冯玉琳;. Recursive Implementation of VLSI Circuits[J]. , 1986, 1(2): 72 -82 .
[4] 章萃; 赵沁平; 徐家福;. Kernel Language KLND[J]. , 1986, 1(3): 65 -79 .
[5] 王建潮; 魏道政;. An Effective Test Generation Algorithm for Combinational Circuits[J]. , 1986, 1(4): 1 -16 .
[6] 陈肇雄; 高庆狮;. A Substitution Based Model for the Implementation of PROLOG——The Design and Implementation of LPROLOG[J]. , 1986, 1(4): 17 -26 .
[7] 黄河燕;. A Parallel Implementation Model of HPARLOG[J]. , 1986, 1(4): 27 -38 .
[8] 郑国梁; 李辉;. The Design and Implementation of the Syntax-Directed Editor Generator(SEG)[J]. , 1986, 1(4): 39 -48 .
[9] 闵应骅; 韩智德;. A Built-in Test Pattern Generator[J]. , 1986, 1(4): 62 -74 .
[10] 黄学东; 蔡莲红; 方棣棠; 迟边进; 周立; 蒋力;. A Computer System for Chinese Character Speech Input[J]. , 1986, 1(4): 75 -83 .
版权所有 © 《计算机科学技术学报》编辑部
本系统由北京玛格泰克科技发展有限公司设计开发 技术支持:support@magtech.com.cn
总访问量: