Facial similarity learning with humans in the loop

Similarity learning has always been a popular topic in computer vision research. Among this, facial similarity is especially important and difficult due to its wide applications and the nonrigid nature of human faces. The large gap between feature representations and human perceptual descriptions makes the problem even harder. In this paper, we learn facial similarity through human-computer interactions. To learn perceptual similarities of faces in a gallery set, we ask users to label some candidate images with their similarities to a probe image. Based on users' responses, a sampling algorithm actively generates a probe image and a set of candidates for the next query. Assisted with human efforts, the algorithm embeds all the images in a space where the distance between two subjects conforms to their dissimilarity in human perception. We apply the learned embedding to face retrieval and compare our method to some feature-based methods on a dataset we collect from social network sites (SNS). Experimental results demonstrate that incorporating human efforts can ensure retrieval accuracy. At the same time, the active sampling algorithm reduces human efforts.