This study introduces a novel conditional recycle generative adversarial network for facial attribute transformation, which can transform high-level semantic face attributes without changing the identity. In our approach, we input a source facial image to the conditional generator with target attribute condition to generate a face with the target attribute. Then we recycle the generated face back to the same conditional generator with source attribute condition. A face should be similar with that of the source face in personal identity and facial attributes is generated. Hence, we introduce a recycle reconstruction loss to enforce the final generated facial image and the source facial image to be identical. Evaluations on the CelebA dataset demonstrate the effectiveness of our approach. Qualitative results show that our method can learn and generate high-quality identity-preserving facial images with specified attributes.
This work was supported by the National Natural Science Foundation of China under Grant Nos. 61672520, 61573348, 61620106003, and 61720106006, the Beijing Natural Science Foundation of China under Grant No. 4162056, the National Key Technology Research and Development Program of China under Grant No. 2015BAH53F02, and the CASIA-Tencent YouTu Jointly Research Project. The Titan X used for this research was donated by the NVIDIA Corporation.
通讯作者: Wei-Ming Dong
About author: Huai-Yu Li is a Ph.D. student in Institute of Automation, Chinese Academy of Sciences, Beijing, under the supervision of Prof. Bao-Gang Hu. He earned his Bachelor's degree in electronic information engineering from Northeast University, Shenyang, in 2014. His research interests are in artificial intelligence, computer vision, and deep learning.
Huai-Yu Li, Wei-Ming Dong, Bao-Gang Hu.基于条件循环生成式对抗网络的人脸图像属性变换[J] Journal of Computer Science and Technology , 2018,V33(3): 511-521
Huai-Yu Li, Wei-Ming Dong, Bao-Gang Hu.Facial Image Attributes Transformation via Conditional Recycle Generative Adversarial Networks[J] Journal of Computer Science and Technology, 2018,V33(3): 511-521
 Selim A, Elgharib M, Doyle L. Painting style transfer for head portraits using convolutional neural networks. ACM Trans. Graph., July 2016, 35(4):129:1-129:18. Kemelmacher-Shlizerman I. Transfiguring portraits. ACM Trans. Graph., 2016, 35(4):94:1-94:8. Li C, Zhou K, Lin S. Simulating makeup through physicsbased manipulation of intrinsic image layers. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2015, pp.4621-4629. Liu S, Ou X, Qian R, Wang W, Cao X. Makeup like a superstar:Deep localized makeup transfer network. In Proc. the 25th International Joint Conference on Artificial Intelligence, July 2016, pp.2568-2575. Tran L, Yin X, Liu X. Disentangled representation learning gan for pose-invariant face recognition. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2017, pp.1283-1292. Chen C, Dantcheva A, Ross A. Automatic facial makeup detection with application in face recognition. In Proc. International Conference on Biometrics, February 2013. Yao N M, Chen H, Guo Q P, Wang H A. Non-frontal facial expression recognition using a depth-patch based deep neural network. Journal of Computer Science and Technology, 2017, 32(6):1172-1185. Mohammed U, Prince S, Kautz J. Visio-lization:Generating novel facial images. ACM Trans. Graph., 2009, 28(3):57:1-57:8. Salimans T, Karpathy A, Chen X, Kingma D P. PixelCNN++:Improving the Pixel-CNN with discretized logistic mixture likelihood and other modifications. In Proc. the 5th Int. Conf. Learning Representations, April 2017. Gregor K, Danihelka I, Graves A, Rezende D, Wierstra D. DRAW:A recurrent neural network for image generation. In Proc. the 32nd International Conference on Machine Learning, July 2015, pp.1462-1471. Kingma D P, Welling M. Auto-encoding variational bayes. In Proc. the 2nd Int. Conf. Learning Representations, April 2014. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, WardeFarley D, Ozair S, Courville A, Bengio Y. Generative adversarial nets. In Proc. the 27th Advances in Neural Information Processing Systems, December 2014, pp.2672-2680. Upchurch P, Gardner J, Bala K, Pless R, Snavely N, Weinberger K. Deep feature interpolation for image content changes. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, July 2016, pp.6090-6099. Perarnau G, Weijer J, Raducanu B, Álvarez J M. Invertible conditional gans for image editing. In Proc. NIPS Workshop on Adversarial Training, December 2016. Yin W, Fu Y, Sigal L, Xue X. Semi-Latent GAN:Learning to generate and modify facial images from attributes. arXiv:1704.02166, 2017. https://arxiv.org/abs/1704.02166, April 2017. Liao J, Yao Y, Yuan L, Hua G, Kang S B. Visual attribute transfer through deep image analogy. ACM Trans. Graph., 2017, 36(4):120:1-120:15. Lu Y, Tai Y W, Tang C K. Conditional CycleGAN for attribute guided face image generation. arXiv:1705.09966, 2017. https://arxiv.org/abs/1705.09966, May 2017. Zhu J Y, Park T, Isola P, Efros A A. Unpaired imageto-image translation using cycle-consistent adversarial networks. In Proc. International Conference on Computer Vision, Oct. 2017, pp.2242-2251. Choi Y, Choi M, Kim M, Ha J W, Kim S, Choo J. Stargan:Unified generative adversarial networks for multidomain image-to-image translation. arXiv:1711.09020, 2017. https://arxiv.org/abs/1711.09020, November 2017. Mirza M, Osindero S. Conditional generative adversarial nets. arXiv:1411.1784, 2014. https://arxiv.org/abs/1411.1784, Mar. 2018. Odena A, Olah C, Shlens J. Conditional image synthesis with auxiliary classifier GANs. In Proc. the 34th International Conference on Machine Learning, August 2017, pp.2642-2651. Isola P, Zhu J Y, Zhou T, Efros A A. Image-to-image translation with conditional adversarial networks. In Proc. Conference on Computer Vision and Pattern Recognition, July 2017, pp.5967-5976. Taigman Y, Polyak A, Wolf L. Unsupervised cross-domain image generation. In Proc. the 5th Int. Conf. Learning Representations, April 2017. Liu Z, Luo P, Wang X, Tang X. Deep learning face attributes in the wild. In Proc. International Conference on Computer Vision, Dec. 2015, pp.3730-3738. Ronneberger O, Fischer P, Brox T et al. U-Net:Convolutional networks for biomedical image segmentation. In Proc. 18th Int. Conf. Medical Image Computing and ComputerAssisted Intervention, October 2015, pp.234-241. Ioffe S, Szegedy C. Batch normalization:Accelerating deep network training by reducing internal covariate shift. In Proc. the 32nd International Conference on Machine Learning, July 2015, pp.448-456. Maas A L, Hannun A Y, Ng A Y. Rectifier nonlinearities improve neural network acoustic models. In Proc. ICML Workshop on Deep Learning for Audio, Speech and Language Processing, June 2013. Zeiler M D, Fergus R. Visualizing and understanding convolutional networks. In Proc. European Conference on Computer Vision, October 2014. Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado G S, Davis A, Dean J, Devin M, Ghemawat S, Goodfellow I, Harp A, Irving G, Isard M, Jia Y, Jozefowicz R, Kaiser L, Kudlur M, Levenberg J, Mané D, Monga R, Moore S, Murray D, Olah C, Schuster M, Shlens J, Steiner B, Sutskever I, Talwar K, Tucker P, Vanhoucke V, Vasudevan V, Viégas F, Vinyals O, Warden P, Wattenberg M, Wicke M, Yu Y, Zheng X. Tensorflow:Large-scale machine learning on heterogeneous distributed systems. arXiv:1603.04467, https://arxiv.org/abs/1603.04467, Mar. 2018. Kingma D, Ba J. Adam:A method for stochastic optimization. In Proc. the 3rd Int. Conf. Learning Representations, May 2015. Yi Z, Zhang H, Tan P, Gong M. DualGAN:Unsupervised dual learning for image-to-image translation. In Proc. International Conference on Computer Vision, Oct. 2017, pp.2868-2876. Kim T, Cha M, Kim H, Lee J K, Kim J. Learning to discover cross-domain relations with generative adversarial networks. In Proc. the 34th International Conference on Machine Learning, August 2017, pp.1857-1865.