SCIE, EI, Scopus, INSPEC, DBLP, CSCD, etc.
Citation: | Li LM, Wang BW, Wang X et al. Face anti-spoofing with unknown attacks: A comprehensive feature extraction and representation perspective. JOURNAL OF COMPUTER SCIENCE AND TECHNOLOGY 39(4): 827−840 July 2024. DOI: 10.1007/s11390-024-4164-7. |
Face anti-spoofing aims at detecting whether the input is a real photo of a user (living) or a fake (spoofing) image. As new types of attacks keep emerging, the detection of unknown attacks, known as Zero-Shot Face Anti-Spoofing (ZSFA), has become increasingly important in both academia and industry. Existing ZSFA methods mainly focus on extracting discriminative features between spoofing and living faces. However, the nature of the spoofing faces is to trick anti-spoofing systems by mimicking the livings, therefore the deceptive features between the known attacks and the livings, which have been ignored by existing ZSFA methods, are essential to comprehensively represent the livings. Therefore, existing ZSFA models are incapable of learning the complete representations of living faces and thus fall short of effectively detecting newly emerged attacks. To tackle this problem, we propose an innovative method that effectively captures both the deceptive and discriminative features distinguishing between genuine and spoofing faces. Our method consists of two main components: a two-against-all training strategy and a semantic autoencoder. The two-against-all training strategy is employed to separate deceptive and discriminative features. To address the subsequent invalidation issue of categorical functions and the dominance disequilibrium issue among different dimensions of features after importing deceptive features, we introduce a modified semantic autoencoder. This autoencoder is designed to map all extracted features to a semantic space, thereby achieving a balance in the dominance of each feature dimension. We combine our method with the feature extraction model ResNet50, and experimental results show that the trained ResNet50 model simultaneously achieves a feasible detection of unknown attacks and comparably accurate detection of known spoofing. Experimental results confirm the superiority and effectiveness of our proposed method in identifying the living with the interference of both known and unknown spoofing types.
[1] |
Li S Z, Jain A K. Handbook of Face Recognition (2nd edition). Springer, 2011.
|
[2] |
Zhao W, Chellappa R, Phillips P J, Rosenfeld A. Face recognition: A literature survey. ACM Computing Surveys, 2003, 35(4): 399–458. DOI: 10.1145/954339.954342.
|
[3] |
Galbally J, Marcel S, Fierrez J. Biometric antispoofing methods: A survey in face recognition. IEEE Access, 2014, 2: 1530–1552. DOI: 10.1109/ACCESS.2014.2381273.
|
[4] |
Chingovska I, Anjos A, Marcel S. On the effectiveness of local binary patterns in face anti-spoofing. In Proc. the International Conference of Biometrics Special Interest Group, Sept. 2012, pp.1–7.
|
[5] |
Best-Rowden L, Han H, Otto C, Klare B F, Jain A K. Unconstrained face recognition: Identifying a person of interest from a media collection. IEEE Trans. Information Forensics and Security, 2014, 9(12): 2144–2157. DOI: 10.1109/TIFS.2014.2359577.
|
[6] |
Wen D, Han H, Jain A K. Face spoof detection with image distortion analysis. IEEE Trans. Information Forensics and Security, 2015, 10(4): 746–761. DOI: 10.1109/TIFS.2015.2400395.
|
[7] |
Boulkenafet Z, Komulainen J, Hadid A. Face spoofing detection using colour texture analysis. IEEE Trans. Information Forensics and Security, 2016, 11(8): 1818–1830. DOI: 10.1109/TIFS.2016.2555286.
|
[8] |
Boulkenafet Z, Komulainen J, Hadid A. Face anti-spoofing based on color texture analysis. In Proc. the 2015 IEEE International Conference on Image Processing, Sept. 2015, pp.2636–2640. DOI: 10.1109/ICIP.2015.7351280.
|
[9] |
Määttä J, Hadid A, Pietikäinen M. Face spoofing detection from single images using micro-texture analysis. In Proc. the 2011 International Joint Conference on Biometrics, Oct. 2011, pp.1–7. DOI: 10.1109/IJCB.2011.6117510.
|
[10] |
Patel K, Han H, Jain A K. Secure face unlock: Spoof detection on smartphones. IEEE Trans. Information Forensics and Security, 2016, 11(10): 2268–2283. DOI: 10.1109/TIFS.2016.2578288.
|
[11] |
Arashloo S R, Kittler J, Christmas W. An anomaly detection approach to face spoofing detection: A new formulation and evaluation protocol. IEEE Access, 2017, 5: 13868–13882. DOI: 10.1109/ACCESS.2017.2729161.
|
[12] |
Nikisins O, Mohammadi A, Anjos A, Marcel S. On effectiveness of anomaly detection approaches against unseen presentation attacks in face anti-spoofing. In Proc. the 2018 International Conference on Biometrics, Feb. 2018, pp.75–81. DOI: 10.1109/ICB2018.2018.00022.
|
[13] |
Liu Y J, Jourabloo A, Liu X M. Learning deep models for face anti-spoofing: Binary or auxiliary supervision. In Proc. the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 2018, pp.389–398. DOI: 10.1109/CVPR.2018.00048.
|
[14] |
Jourabloo A, Liu Y J, Liu X M. Face de-spoofing: Anti-spoofing via noise modeling. In Proc. the 15th European Conference on Computer Vision, Sept. 2018, pp.297–315. DOI: 10.1007/978-3-030-01261-8_18.
|
[15] |
Atoum Y, Liu Y J, Jourabloo A, Liu X M. Face anti-spoofing using patch and depth-based CNNs. In Proc. the 2017 IEEE International Joint Conference on Biometrics, Oct. 2017, pp.319–328. DOI: 10.1109/BTAS.2017.8272713.
|
[16] |
Feng L T, Po L M, Li Y M, Xu X Y, Yuan F, Cheung T C H, Cheung K W. Integration of image quality and motion cues for face anti-spoofing: A neural network approach. Journal of Visual Communication and Image Representation, 2016, 38: 451–460. DOI: 10.1016/j.jvcir.2016.03.019.
|
[17] |
Xiong F, AbdAlmageed W. Unknown presentation attack detection with face RGB images. In Proc. the 9th IEEE International Conference on Biometrics Theory, Applications and Systems, Oct. 2018, pp.1–9. DOI: 10.1109/BTAS.2018.8698574.
|
[18] |
Liu Y J, Stehouwer J, Jourabloo A, Liu X M. Deep tree learning for zero-shot face anti-spoofing. In Proc. the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 2019, pp.4675–4684. DOI: 10.1109/CVPR.2019.00481.
|
[19] |
Liu S C, Lu S T, Xu H Y, Yang J, Ding S H, Ma L Z. Feature generation and hypothesis verification for reliable face anti-spoofing. In Proc. the 36th AAAI Conference on Artificial Intelligence, Jun. 2022, pp.1782–1791. DOI: 10.1609/AAAI.V36I2.20071.
|
[20] |
Liu Y J, Stehouwer J, Liu X M. On disentangling spoof trace for generic face anti-spoofing. In Proc. the 16th European Conference on Computer Vision, Aug. 2020, pp.406–422. DOI: 10.1007/978-3-030-58523-5_24.
|
[21] |
Yu Z T, Qin Y X, Zhao H S, Li X B, Zhao G Y. Dual-cross central difference network for face anti-spoofing. In Proc. the 30th International Joint Conference on Artificial Intelligence, Aug. 2021, pp.1281–1287. DOI: 10.24963/IJCAI.2021/177.
|
[22] |
Qin Y X, Zhao C X, Zhu X Y, Wang Z Z, Yu Z T, Fu T Y, Zhou F, Shi J P, Lei Z. Learning meta model for zero- and few-shot face anti-spoofing. In Proc. the 34th AAAI Conference on Artificial Intelligence, Feb. 2020, pp.11916–11923. DOI: 10.1609/AAAI.V34I07.6866.
|
[23] |
Kodirov E, Xiang T, Gong S G. Semantic autoencoder for zero-shot learning. In Proc. the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Jul. 2017, pp.4447–4456. DOI: 10.1109/CVPR.2017.473.
|
[24] |
Wang C Y, Lu Y D, Yang S T, Lai S H. Patchnet: A simple face anti-spoofing framework via fine-grained patch recognition. In Proc. the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 2022, pp.20249–20258. DOI: 10.1109/CVPR52688.2022.01964.
|
[25] |
Wang Z, Wang Z Z, Yu Z T, Deng W H, Li J H, Gao T T, Wang Z Y. Domain generalization via shuffled style assembly for face anti-spoofing. In Proc. the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 2022, pp.4113–4123. DOI: 10.1109/CVPR52688.2022.00409.
|
[26] |
Liu S Q, Yang B Y, Yuen P C, Zhao G Y. A 3D mask face anti-spoofing database with real world variations. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops, Jun. 26–Jul. 1, 2016, pp.1551–1557. DOI: 10.1109/CVPRW.2016.193.
|
[27] |
He K M, Zhang X Y, Ren S Q, Sun J. Deep residual learning for image recognition. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2016, pp.770–778. DOI: 10.1109/CVPR.2016.90.
|
[28] |
Tibshirani R. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Methodological), 1996, 58(1): 267–288. DOI: 10.1111/j.2517-6161.1996.tb02080.x.
|
[29] |
Wen Y D, Zhang K P, Li Z F, Qiao Y. A discriminative feature learning approach for deep face recognition. In Proc. the 14th European Conference on Computer Vision, Oct. 2016, pp.499–515. DOI: 10.1007/978-3-319-46478-7_31.
|
[30] |
Van Der Maaten L, Hinton G. Visualizing data using t-SNE. Journal of Machine Learning Research, 2008, 9(86): 2579–2605.
|
[31] |
Ranzato M A, Boureau Y L, LeCun Y. Sparse feature learning for deep belief networks. In Proc. the 20th International Conference on Neural Information Processing Systems, Dec. 2007, pp.1185–1192.
|
[32] |
Boulkenafet Z, Komulainen J, Li L, Feng X Y, Hadid A. OULU-NPU: A mobile face presentation attack database with real-world variations. In Proc. the 12th IEEE International Conference on Automatic Face & Gesture Recognition, May 30–Jun. 3, 2017, pp.612-618. DOI: 10.1109/FG.2017.77.
|
[33] |
Zhang Z W, Yan J J, Liu S F, Lei Z, Yi D, Li S Z. A face antispoofing database with diverse attacks. In Proc. the 5th IAPR International Conference on Biometrics, Mar. 29–Apr. 1, 2012, pp.26-31. DOI: 10.1109/ICB.2012.6199754.
|
[34] |
George A, Mostaani Z, Geissenbuhler D, Nikisins O, Anjos A, Marcel S. Biometric face presentation attack detection with multi-channel convolutional neural network. IEEE Trans. Information Forensics and Security, 2020, 15: 42–55. DOI: 10.1109/TIFS.2019.2916652.
|
[35] |
Froba B, Ernst A. Face detection with the modified census transform. In Proc. the 6th IEEE International Conference on Automatic Face and Gesture Recognition, May 2004, pp.91–96. DOI: 10.1109/AFGR.2004.1301514.
|
[36] |
Kingma D P, Ba J. Adam: A method for stochastic optimization. In Proc. the 3rd International Conference on Learning Representations, May 2015.
|
[1] | Xin Zhang, Siyuan Lu, Shui-Hua Wang, Xiang Yu, Su-Jing Wang, Lun Yao, Yi Pan, Yu-Dong Zhang. Diagnosis of COVID-19 Pneumonia via a Novel Deep Learning Architecture[J]. Journal of Computer Science and Technology, 2022, 37(2): 330-343. DOI: 10.1007/s11390-020-0679-8 |
[2] | Dan-Hao Zhu, Xin-Yu Dai, Jia-Jun Chen. Pre-Train and Learn: Preserving Global Information for Graph Neural Networks[J]. Journal of Computer Science and Technology, 2021, 36(6): 1420-1430. DOI: 10.1007/s11390-020-0142-x |
[3] | Wei Du, Yu Sun, Hui-Min Bao, Liang Chen, Ying Li, Yan-Chun Liang. DeepHBSP: A Deep Learning Framework for Predicting Human Blood-Secretory Proteins Using Transfer Learning[J]. Journal of Computer Science and Technology, 2021, 36(2): 234-247. DOI: 10.1007/s11390-021-0851-9 |
[4] | Robail Yasrab. SRNET: A Shallow Skip Connection Based Convolutional Neural Network Design for Resolving Singularities[J]. Journal of Computer Science and Technology, 2019, 34(4): 924-938. DOI: 10.1007/s11390-019-1950-8 |
[5] | Dong-Di Zhao, Fan Li, Kashif Sharif, Guang-Min Xia, Yu Wang. Space Efficient Quantization for Deep Convolutional Neural Networks[J]. Journal of Computer Science and Technology, 2019, 34(2): 305-317. DOI: 10.1007/s11390-019-1912-1 |
[6] | Nai-Ming Yao, Hui Chen, Qing-Pei Guo, Hong-An Wang. Non-Frontal Facial Expression Recognition Using a Depth-Patch Based Deep Neural Network[J]. Journal of Computer Science and Technology, 2017, 32(6): 1172-1185. DOI: 10.1007/s11390-017-1792-1 |
[7] | Xu-Ran Zhao, Xun Wang, Qi-Chao Chen. Temporally Consistent Depth Map Prediction Using Deep CNN and Spatial-temporal Conditional Random Field[J]. Journal of Computer Science and Technology, 2017, 32(3): 443-456. DOI: 10.1007/s11390-017-1735-x |
[8] | Hui-Ying Lan, Lin-Yang Wu, Xiao Zhang, Jin-Hua Tao, Xun-Yu Chen, Bing-Rui Wang, Yu-Qing Wang, Qi Guo, Yun-Ji Chen. DLPlib: A Library for Deep Learning Processor[J]. Journal of Computer Science and Technology, 2017, 32(2): 286-296. DOI: 10.1007/s11390-017-1722-2 |
[9] | Xi-Jin Zhang, Yi-Fan Lu, Song-Hai Zhang. Multi-Task Learning for Food Identification and Analysis with Deep Convolutional Neural Networks[J]. Journal of Computer Science and Technology, 2016, 31(3): 489-500. DOI: 10.1007/s11390-016-1642-6 |
[10] | Yao Shu, Zhang Bo. The Learning Convergence of CMAC in Cyclic Learning[J]. Journal of Computer Science and Technology, 1994, 9(4): 320-328. |