›› 2015,Vol. 30 ›› Issue (3): 499-510.doi: 10.1007/s11390-015-1540-3

所属专题: Artificial Intelligence and Pattern Recognition Computer Graphics and Multimedia

• Special Section on Selected Paper from NPC 2011 • 上一篇    下一篇

用户参与的人脸相似度学习

Chong Cao(曹翀), Student Member, CCF, Hai-Zhou Ai(艾海舟), Senior Member, IEEE, Member, CCF   

  1. Tsinghua National Laboratory for Information Science and Technology, Department of Computer Science and Technology Tsinghua University, Beijing 100084, China
  • 收稿日期:2014-11-28 修回日期:2015-03-07 出版日期:2015-05-05 发布日期:2015-05-05
  • 作者简介:Chong Cao received her B.S. degree in computer science and technology from Tsinghua University, Beijing, in 2010. She is currently a Ph.D. candidate at Tsinghua University. Her research interests include computer vision, pattern recognition and multimedia, with special focus on face retrieval and facial similarity learning.
  • 基金资助:

    This work was partly supported by the National Basic Research 973 Program of China under Grant No. 2011CB302203.

Facial similarity learning with humans in the loop

Chong Cao(曹翀), Student Member, CCF, Hai-Zhou Ai(艾海舟), Senior Member, IEEE, Member, CCF   

  1. Tsinghua National Laboratory for Information Science and Technology, Department of Computer Science and Technology Tsinghua University, Beijing 100084, China
  • Received:2014-11-28 Revised:2015-03-07 Online:2015-05-05 Published:2015-05-05
  • About author:Chong Cao received her B.S. degree in computer science and technology from Tsinghua University, Beijing, in 2010. She is currently a Ph.D. candidate at Tsinghua University. Her research interests include computer vision, pattern recognition and multimedia, with special focus on face retrieval and facial similarity learning.
  • Supported by:

    This work was partly supported by the National Basic Research 973 Program of China under Grant No. 2011CB302203.

相似度学习一直都是计算机视觉研究中的热门话题.其中,人脸相似度由于其广泛的应用背景以及多变性而显得尤为重要和困难.在特征表述与人类感知上对人脸的描述之间仍旧有着难以逾越的鸿沟.本文通过人机交互来学习基于人类感知的人脸相似度.为达成这个目标,我们首先让用户来标注一些查询图像和候选图像集之间的相似程度.然后,通过采样算法产生后续标注所需的查询图像和候选图像集.这样,我们将所有图像嵌入到一个和人类感知相符的空间.我们从社交网站上收集了一个人脸数据集,在嵌入空间上进行人脸检索并与其它基于特征的方法进行比较.实验结果显示用户标注可以保证我们的检索准确度.与此同时,文中提出的采样算法可以有效减少用户的工作量.

Abstract: Similarity learning has always been a popular topic in computer vision research. Among this, facial similarity is especially important and difficult due to its wide applications and the nonrigid nature of human faces. The large gap between feature representations and human perceptual descriptions makes the problem even harder. In this paper, we learn facial similarity through human-computer interactions. To learn perceptual similarities of faces in a gallery set, we ask users to label some candidate images with their similarities to a probe image. Based on users' responses, a sampling algorithm actively generates a probe image and a set of candidates for the next query. Assisted with human efforts, the algorithm embeds all the images in a space where the distance between two subjects conforms to their dissimilarity in human perception. We apply the learned embedding to face retrieval and compare our method to some feature-based methods on a dataset we collect from social network sites (SNS). Experimental results demonstrate that incorporating human efforts can ensure retrieval accuracy. At the same time, the active sampling algorithm reduces human efforts.

[1] Yuen P C, Man C. Human face image searching system using sketches. IEEE Trans. Systems, Man and Cybernetics, Part A: Systems and Humans, 2007, 37(4): 493-504.

[2] Kumar N, Berg A C, Belhumeur P N, Nayar S K. Attribute and simile classifiers for face verification. In Proc. the 12th IEEE International Conference on Computer Vision (ICCV), Sept. 29-Oct. 2, 2009, pp.365-372.

[3] Jain A K, Vailaya A. Image retrieval using color and shape. Pattern Recognition, 1996, 29(8): 1233-1244.

[4] Ahonen T, Hadid A, Pietikäinen M. Face description with local binary patterns: Application to face recognition. IEEE Trans. Pattern Analysis and Machine Intelligence, 2006, 28(12): 2037-2041.

[5] Wiskott L, Fellous J M, Krüger N et al. Face recognition by elastic bunch graph matching. IEEE Trans. Pattern Analysis and Machine Intelligence, 1997, 19(7): 775-779.

[6] Berg T, Belhumeur P N. Tom-vs-pete classifiers and identity-preserving alignment for face verification. In Proc. the British Machine Vision Conference, September 2012.

[7] Chopra S, Hadsell R, LeCun Y. Learning a similarity metric discriminatively, with application to face verification. In Proc. the IEEE Conf. Computer Vision and Pattern Recognition (CVPR), June 2005, pp.539-546.

[8] Schroff F, Treibitz T, Kriegman D, Belongie S. Pose, illumination and expression invariant pairwise face-similarity measure via doppelgänger list comparison. In Proc. the 13th IEEE ICCV, Nov. 2011, pp.2494-2501.

[9] Yin Q, Tang X, Sun J. An associate-predict model for face recognition. In Proc. the IEEE CVPR, June 2011, pp.497-504.

[10] Scheirer W J, Kumar N, Belhumeur P N, Boult T E. Multiattribute spaces: Calibration for attribute fusion and similarity search. In Proc. the IEEE CVPR, June 2012, pp.2933-2940.

[11] Siddiquie B, Feris R S, Davis L S. Image ranking and retrieval based on multi-attribute queries. In Proc. the IEEE CVPR, June 2011, pp.801-808.

[12] Parikh D, Grauman K. Relative attributes. In Proc. the 13th IEEE Int. Conf. Computer Vision, Nov. 2011, pp.503-510.

[13] Biswas A, Parikh D. Simultaneous active learning of classifiers & attributes via relative feedback. In Proc. the IEEE CVPR, June 2013, pp.644-651.

[14] Liang L, Grauman K. Beyond comparing image pairs: Setwise active learning for relative attributes. In Proc. the IEEE CVPR, June 2014, pp.208-215.

[15] Agarwal S, Wills J, Cayton L et al. Generalized non-metric multidimensional scaling. In Proc. the 11th Int. Conf. Artificial Intelligence and Statistics, March 2007, pp.11-18.

[16] Tamuz O, Liu C, Belongie S, Shamir O, Kalai A T. Adaptively learning the crowd kernel. arXiv:1105.1033, 2011. http://arxiv.org/abs/1105.1033, March 2015.

[17] van der Maaten L, Weinberger K. Stochastic triplet embedding. In Proc. IEEE International Workshop on Machine Learning for Signal Processing, September 2012.

[18] Garces E, Agarwala A, Gutierrez D, Hertzmann A. A similarity measure for illustration style. ACM Transactions on Graphics, 2014, 33(4): 93:1-93:9.

[19] Deng J, Krause J, Li F F. Fine-grained crowdsourcing for fine-grained recognition. In Proc. IEEE CVPR, June 2013, pp.580-587.

[20] Holub A, Liu Y H, Perona P. On constructing facial similarity maps. In Proc. IEEE CVPR, June 2007.

[21] Kruskal J B. Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis. Psychometrika, 1964, 29(1): 1-27.

[22] Fang Y, Geman D. Experiments in mental face retrieval. In Lecture Notes in Computer Science Volume 3546, Kanade T, Jain A, Ratha N K (eds.), Springer, 2005, pp.637-646.

[23] Ruiz-del-Solar J, Navarrete P. FACERET: An interactive face retrieval system based on self-organizing maps. In Proc. Int. Conf. Image and Video Retrieval, July 2002, pp.157-164.

[24] Yang Z, Laaksonen J. Partial relevance in interactive facial image retrieval. In Proc. the 3rd Int. Conf. Advances in Pattern Recognition, Part II, August 2005, pp.216-225.

[25] Cao C, Kwak S, Belongie S, Kriegman D, Ai H. Adaptive ranking of facial attractiveness. In Proc. the IEEE International Conference on Multimedia and Expo, July 2014.

[26] Huang G B, Ramesh M, Berg T, Learned-Miller E. Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical Report 07-49, University of Massachusetts, Amherst, Oct. 2007.

[27] Huang C, Ai H, Li Y, Lao S. High-performance rotation invariant multiview face detection. IEEE Trans. Pattern Analysis and Machine Intelligence, 2007, 29(4): 671-686.

[28] Yan D, Huang L, Jordan M I. Fast approximate spectral clustering. In Proc. the 15th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, June 2009, pp.907-916.
No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] 刘明业; 洪恩宇;. Some Covering Problems and Their Solutions in Automatic Logic Synthesis Systems[J]. , 1986, 1(2): 83 -92 .
[2] 陈世华;. On the Structure of (Weak) Inverses of an (Weakly) Invertible Finite Automaton[J]. , 1986, 1(3): 92 -100 .
[3] 高庆狮; 张祥; 杨树范; 陈树清;. Vector Computer 757[J]. , 1986, 1(3): 1 -14 .
[4] 陈肇雄; 高庆狮;. A Substitution Based Model for the Implementation of PROLOG——The Design and Implementation of LPROLOG[J]. , 1986, 1(4): 17 -26 .
[5] 黄河燕;. A Parallel Implementation Model of HPARLOG[J]. , 1986, 1(4): 27 -38 .
[6] 闵应骅; 韩智德;. A Built-in Test Pattern Generator[J]. , 1986, 1(4): 62 -74 .
[7] 唐同诰; 招兆铿;. Stack Method in Program Semantics[J]. , 1987, 2(1): 51 -63 .
[8] 闵应骅;. Easy Test Generation PLAs[J]. , 1987, 2(1): 72 -80 .
[9] 朱鸿;. Some Mathematical Properties of the Functional Programming Language FP[J]. , 1987, 2(3): 202 -216 .
[10] 李明慧;. CAD System of Microprogrammed Digital Systems[J]. , 1987, 2(3): 226 -235 .
版权所有 © 《计算机科学技术学报》编辑部
本系统由北京玛格泰克科技发展有限公司设计开发 技术支持:support@magtech.com.cn
总访问量: