? Non-Frontal Facial Expression Recognition Using a Depth-Patch Based Deep Neural Network
Journal of Computer Science and Technology
Quick Search in JCST
 Advanced Search 
      Home | PrePrint | SiteMap | Contact Us | FAQ
 
Indexed by   SCIE, EI ...
Bimonthly    Since 1986
Journal of Computer Science and Technology 2017, Vol. 32 Issue (6) :1172-1185    DOI: 10.1007/s11390-017-1792-1
Special Section of CAD/Graphics 2017 Current Issue | Archive | Adv Search << Previous Articles | Next Articles >>
Non-Frontal Facial Expression Recognition Using a Depth-Patch Based Deep Neural Network
Nai-Ming Yao1,2, Hui Chen1,2,*, Member, CCF, Qing-Pei Guo1,2, Hong-An Wang1,2,3, Member, CCF, IEEE
1 Beijing Key Laboratory of Human-Computer Interaction, Institute of Software, Chinese Academy of Sciences Beijing 100190, China;
2 University of Chinese Academy of Sciences, Beijing 100049, China;
3 State Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences, Beijing 100190, China

Abstract
Reference
Related Articles
Download: [PDF 1698KB]     Export: BibTeX or EndNote (RIS)  
Abstract The challenge of coping with non-frontal head poses during facial expression recognition results in considerable reduction of accuracy and robustness when capturing expressions that occur during natural communications. In this paper, we attempt to recognize facial expressions under poses with large rotation angles from 2D videos. A depth-patch based 4D expression representation model is proposed. It was reconstructed from 2D dynamic images for delineating continuous spatial changes and temporal context under non-frontal cases. Furthermore, we present an effective deep neural network classifier, which can accurately capture pose-variant expression features from the depth patches and recognize non-frontal expressions. Experimental results on the BU-4DFE database show that the proposed method achieves a high recognition accuracy of 86.87% for non-frontal facial expressions within a range of head rotation angle of up to 52°, outperforming existing methods. We also present a quantitative analysis of the components contributing to the performance gain through tests on the BU-4DFE and Multi-PIE datasets.
Articles by authors
Keywordsfacial expression recognition   non-frontal head pose   depth   spatial-temporal   convolutional neural network     
Received 2017-06-20;
Fund:

This work was supported by the National Key Research and Development Program of China under Grant No. 2016YFB1001405, and the National Natural Science Foundation of China under Grant Nos. 61232013, 61422212, and 61661146002.

Corresponding Authors: Hui Chen     Email: chenhui@iscas.ac.cn
About author: Nai-Ming Yao is a Ph.D.candidate at Institute of Software,Chinese Academy of Sciences,Beijing,and University of Chinese Academy of Sciences,Beijing.His research interests include human-computer interaction,affective computing,machine learning,and computer vision.
Cite this article:   
Nai-Ming Yao, Hui Chen, Qing-Pei Guo, Hong-An Wang.Non-Frontal Facial Expression Recognition Using a Depth-Patch Based Deep Neural Network[J]  Journal of Computer Science and Technology, 2017,V32(6): 1172-1185
URL:  
http://jcst.ict.ac.cn:8080/jcst/EN/10.1007/s11390-017-1792-1
Copyright 2010 by Journal of Computer Science and Technology