Journal of Computer Science and Technology ›› 2019, Vol. 34 ›› Issue (1): 47-60.doi: 10.1007/s11390-019-1898-8

Special Issue: Artificial Intelligence and Pattern Recognition

• Special Section of Advances in Computer Science and Technology—Current Advances in the NSFC Joint Research Fund for Overseas Chinese Scholars and Scholars in Hong Kong and Macao 2014-2017 (Part 1) • Previous Articles     Next Articles

Privacy-Protective-GAN for Privacy Preserving Face De-Identification

Yifan Wu1, Fan Yang1, Yong Xu2, Senior Member, CCF, ACM, IEEE, and Haibin Ling1, Senior Member, IEEE   

  1. 1 Department of Computer and Information Sciences, Temple University, Philadelphia 19122, U.S.A.;
    2 School of Computer Science and Engineering, South China University of Technology, Guangzhou 510006, China
  • Received:2018-07-12 Revised:2018-12-20 Online:2019-01-05 Published:2019-01-12
  • About author:Yifan Wu received her B.S. degree in communication engineering from China University of Geosciences, Wuhan, and M.S. degree in computer science from Temple University, Philadelphia, in 2016 and 2017, respectively. She is currently a Ph.D. student in bioengineering at University of Pennsylvania, Philadelphia. Her current research interests are computer vision and medical image processing.
  • Supported by:
    This work was supported by the National Natural Science Foundation of China under Grant No. 61528204, and the National Science Foundation of USA under Grant No. 1350521.

PDF

974

Supplement

1

Face de-identification has become increasingly important as the image sources are explosively growing and easily accessible. The advance of new face recognition techniques also arises people's concern regarding the privacy leakage. The mainstream pipelines of face de-identification are mostly based on the k-same framework, which bears critiques of low effectiveness and poor visual quality. In this paper, we propose a new framework called Privacy-Protective-GAN (PP-GAN) that adapts GAN (generative adversarial network) with novel verificator and regulator modules specially designed for the face de-identification problem to ensure generating de-identified output with retained structure similarity according to a single input. We evaluate the proposed approach in terms of privacy protection, utility preservation, and structure similarity. Our approach not only outperforms existing face de-identification techniques but also provides a practical framework of adapting GAN with priors of domain knowledge.

Key words: face de-identification; privacy protection; image synthesis; generative adversarial network (GAN);

[1] Newton E M, Sweeney L, Malin B. Preserving privacy by de-identifying face images. IEEE Transactions on Knowledge and Data Engineering, 2005, 17(2):232-243.
[2] Ribaric S, Ariyaeeinia A, Pavesic N. De-identification for privacy protection in multimedia content:A survey. Signal Processing:Image Communication, 2016, 47:131-151.
[3] Gross R, Airoldi E, Malin B, Sweeney L. Integrating utility into face de-identification. In Proc. the 5th International Conference on Privacy Enhancing Technologies, May 2005, pp.227-242.
[4] Gross R, Sweeney L, de la Torre F, Baker S. Model-based face de-identification. In Proc. the 2006 Conference on Computer Vision and Pattern Recognition, Jun. 2006, Article No. 161.
[5] Gross R, Sweeney L, de La Torre F, Baker S. Semisupervised learning of multi-factor models for face deidentification. In Proc. the 2008 Conference on Computer Vision and Pattern Recognition, Jun. 2008, Article No. 10.
[6] Matthews I, Baker S. Active appearance models revisited. International Journal of Computer Vision, 2004, 60(2):135-164.
[7] Du L, Yi M, Blasch E, Ling H. GARP-face:Balancing privacy protection and utility preservation in face deidentification. In Proc. IEEE International Joint Conference on Biometrics, Sept. 2014, Article No. 67.
[8] Jourabloo A, Yin X, Liu X. Attribute preserved face deidentification. In Proc. IEEE International Conference on Biometrics, May 2015, pp.278-285.
[9] Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, WardeFarley D, Ozair S, Courville A, Bengio Y. Generative adversarial nets. In Proc. Annual Conference on Neural Information Processing Systems, Dec. 2014, pp.2672-2680.
[10] Isola P, Zhu J Y, Zhou T, Efros A A. Image-to-image translation with conditional adversarial networks. In Proc. the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Jul. 2017, pp.5967-5976.
[11] Zhu J Y, Park T, Isola P, Efros A A. Unpaired imageto-image translation using cycle-consistent adversarial networks. In Proc. the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Jul. 2017, pp.2242-2251.
[12] Hadsell R, Chopra S, LeCun Y. Dimensionality reduction by learning an invariant mapping. In Proc. the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Jun. 2006, pp.1735-1742.
[13] Wang Z, Bovik A C, Sheikh H R, Simoncelli E P. Image quality assessment:From error visibility to structural similarity. IEEE Transactions on Image Processing, 2004, 13(4):600-612.
[14] Schroff F, Kalenichenko D, Philbin J. FaceNet:A unified embedding for face recognition and clustering. In Proc. the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2015, pp.815-823.
[15] Zhang K, Zhang Z, Li Z, Qiao Y. Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Processing Letters, 2016, 23(10):1499-1503.
[16] Sun Q, Tewari A, Xu W, Fritz M, Theobalt C, Schiele B. A hybrid model for identity obfuscation by face replacement. In Proc. the 15th European Conference on Computer Vision, Sept. 2018, pp.570-586.
[17] Chen D, Chang Y, Yan R, Yang J. Tools for protecting the privacy of specific individuals in video. EURASIP Journal on Advances in Signal Processing, 2007, 2007(1):Article No. 075427.
[18] Wilber M J, Boult T E. Secure remote matching with privacy:Scrambled support vector vaulted verification (S2V3). In Proc. IEEE Workshop on Applications of Computer Vision, Jan. 2012, pp.169-176.
[19] Chan A B, Liang Z S, Vasconcelos N. Privacy preserving crowd monitoring:Counting people without people models or tracking. In Proc. the 2008 IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2018, Article No. 229.
[20] Sun Q, Ma L, Oh S J, van Gool L, Schiele B, Fritz M. Natural and effective obfuscation by head inpainting. In Proc. the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 2018, pp.5050-5059.
[21] Sun Q, Schiele B, Fritz M. A domain based approach to social relation recognition. In Proc. the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Jul. 2017, pp.435-444.
[22] Mirjalili V, Raschka S, Namboodiri A, Ross A. Semiadversarial networks:Convolutional autoencoders for imparting privacy to face images. In Proc. the 2018 International Conference on Biometrics, Feb. 2018 pp.82-89.
[23] Ribaric S, Ariyaeeinia A, Pavesic N. De-identification for privacy protection in multimedia content:A survey. Signal Processing:Image Communication, 2016, 47:131-151.
[24] Gross R, Sweeney L, Cohn J, de la Torre F, Baker S. Face de-identification. In Protecting Privacy in Video Surveillance, Senior A (ed.), Springer, 2009, pp.129-146.
[25] Frome A, Cheung G, Abdulkader A, Zennaro M, Wu B, Bissacco A, Adam H, Neven H, Vincent L. Large-scale privacy protection in Google street view. In Proc. the 12th International Conference on Computer Vision, Sept. 2009, pp.2373-2380.
[26] Sweeney L. k-anonymity:A model for protecting privacy. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 2002, 10(5):557-570.
[27] Brkic K, Sikiric I, Hrkac T, Kalafatic Z. I know that person:Generative full body and face de-identification of people. In Proc. the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops, Jul. 2017, pp.1319-1328.
[28] Meden B, Mallı R C, Fabijan S, Ekenel H K, Struc V, Peer P. Face deidentification with generative deep neural networks. IET Signal Processing, 2017, 11(9):1046-54.
[29] Meden B, Ziga E, Struc V, Peer P. k-same-net:k-anonymity with generative deep neural networks for face deidentification. Entropy, 2018, 20(1):Article No. 60.
[30] Ioffe S, Szegedy C. Batch normalization:Accelerating deep network training by reducing internal covariate shift. In Proc. the 32nd International Conference on Machine Learning, Jul. 2015, pp.448-456.
[31] Ronneberger O, Fischer P, Brox T. U-net:Convolutional networks for biomedical image segmentation. In Proc. the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention, Oct. 2015 pp.234-241.
[32] Chopra S, Hadsell R, LeCun Y. Learning a similarity metric discriminatively, with application to face verification. In Proc. the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Jun. 2005, pp.539-546.
[33] Sun Y, Chen Y, Wang X, Tang X. Deep learning face representation by joint identification-verification. In Proc. the 2014 Annual Conference on Neural Information Processing Systems, Dec. 2014, pp.1988-1996.
[34] Schroff F, Kalenichenko D, Philbin J. FaceNet:A unified embedding for face recognition and clustering. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2015, pp.815-823.
[35] Parkhi O M, Vedaldi A, Zisserman A. Deep face recognition. In Proc. the 26th British Machine Vision Conference, Sept. 2015, Article No. 6.
[36] Wu X, He R, Sun Z, Tan T. A light CNN for deep face representation with noisy labels. IEEE Transactions on Information Forensics and Security, 2018, 13(11):2884-2896.
[37] Lin M, Chen Q, Yan S. Network in network. arXiv:1312.4400, 2013. https://arxiv.org/pdf/131-2.4400.pdf, Sept. 2018.
[38] Du L, Ling H. Preservative license plate de-identification for privacy protection. In Proc. the 2011 International Conference on Document Analysis and Recognition, Sept. 2011, pp.468-472.
[39] Ricanek K, Tesafaye T. MORPH:A longitudinal image database of normal adult age-progression. In Proc. the 7th International Conference on Automatic Face and Gesture Recognition, Apr. 2006, pp.341-345.
[40] Kingma D P, Ba J. Adam:A method for stochastic optimization. arXiv:1412.6980, 2015. https://arxiv.org/pdf/1412.6980.pdf, Sept. 2018.
[1] Fei Fang, Fei Luo, Hong-Pan Zhang, Hua-Jian Zhou, Alix L. H. Chow, Chun-Xia Xiao. A Comprehensive Pipeline for Complex Text-to-Image Synthesis [J]. Journal of Computer Science and Technology, 2020, 35(3): 522-537.
[2] Chong Wang, Nasro Min-Allah, Bei Guan, Yu-Qi Lin, Jing-Zheng Wu, Yong-Ji Wang. An Efficient Approach for Mitigating Covert Storage Channel Attacks in Virtual Machines by the Anti-Detection Criterion [J]. Journal of Computer Science and Technology, 2019, 34(6): 1351-1365.
[3] Zi-Peng Zhang, Ming Fu, Xin-Yu Feng. A Lightweight Dynamic Enforcement of Privacy Protection for Android [J]. Journal of Computer Science and Technology, 2019, 34(4): 901-923.
[4] Lei Cui, Youyang Qu, Mohammad Reza Nosouhi, Shui Yu, Jian-Wei Niu, Gang Xie. Improving Data Utility Through Game Theory in Personalized Differential Privacy [J]. Journal of Computer Science and Technology, 2019, 34(2): 272-286.
[5] Yu-Tao Liu, Dong Du, Yu-Bin Xia, Hai-Bo Chen, Bin-Yu Zang, Zhenkai Liang. SplitPass: A Mutually Distrusting Two-Party Password Manager [J]. , 2018, 33(1): 98-115.
[6] An Liu, Zhi-Xu Li, Guan-Feng Liu, Kai Zheng, Min Zhang, Qing Li, Xiangliang Zhang. Privacy-preserving Task Assignment in Spatial Crowdsourcing [J]. , 2017, 32(5): 905-918.
[7] ZHANG Xiaopeng（张晓鹏），CHEN Yanyun（陈彦云）and WU Enhua（吴恩华）. Hair Image Generation Using Connected Texels [J]. , 2001, 16(4): 0-0.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!

ISSN 1000-9000(Print)

         1860-4749(Online)
CN 11-2296/TP

 Home
Editorial Board
Author Guidelines
Subscription		 Journal of Computer Science and Technology
Institute of Computing Technology, Chinese Academy of Sciences
P.O. Box 2704, Beijing 100190 P.R. China
Tel.:86-10-62610746
E-mail: jcst@ict.ac.cn
 
  Copyright ©2015 JCST, All Rights Reserved