? Improving the Quality of Crowdsourced Image Labeling via Label Similarity
Journal of Computer Science and Technology
Quick Search in JCST
 Advanced Search 
      Home | PrePrint | SiteMap | Contact Us | FAQ
 
Indexed by   SCIE, EI ...
Bimonthly    Since 1986
Journal of Computer Science and Technology 2017, Vol. 32 Issue (5) :877-889    DOI: 10.1007/s11390-017-1770-7
Special Section on Crowdsourced Data Management Current Issue | Archive | Adv Search << Previous Articles | Next Articles >>
Improving the Quality of Crowdsourced Image Labeling via Label Similarity
Yi-Li Fang1,2, Hai-Long Sun1,2,*, Member, CCF, ACM, IEEE, Peng-Peng Chen1,2, Ting Deng1,2, Member, CCF, ACM
1 State Key Laboratory of Software Development Environment, Beihang University, Beijing 100191, China;
2 School of Computer Science and Engineering, Beihang University, Beijing 100191, China

Abstract
Reference
Related Articles
Download: [PDF 1205KB]     Export: BibTeX or EndNote (RIS)  
Abstract Crowdsourcing is an effective method to obtain large databases of manually-labeled images, which is especially important for image understanding with supervised machine learning algorithms. However, for several kinds of tasks regarding image labeling, e.g. dog breed recognition, it is hard to achieve high-quality results. Therefore, further optimizing crowdsourcing workflow mainly involves task allocation and result inference. In task allocation, we design a two-round crowdsourcing framework, which contains a smart decision mechanism based on information entropy to determine whether to perform a second round task allocation. Regarding result inference, after quantifying the similarity of all labels,two graphical models are proposed to describe the labeling process and corresponding inference algorithms are designed to further improve the result quality of image labeling. Extensive experiments on real-world tasks in Crowdflower and synthesis datasets were conducted. The experimental results demonstrate the superiority of these approaches in comparison with state-of-the-art methods.
Articles by authors
KeywordsImage Labeling   Crowdsourcing   Information Entropy   Label Similarity     
Received 2017-03-02;
Fund:

This work was supported partly by the National Key Research and Development Program of China under Grant No. 2016YFB1000804, the National Natural Science Foundation of China under Grant No. 61602023, the National Basic Research 973 Program of China under Grant Nos. 2014CB340304 and 2015CB358700, and the State Key Laboratory of Software Development Environment under Grant No. SKLSDE-2017ZX-14.

Corresponding Authors: Hai-Long Sun,sunhl@act.buaa.edu.cn     Email: sunhl@act.buaa.edu.cn
About author: Yi-Li Fang is a Ph.D. student in the School of Computer Science and Engineering, Beihang University, Beijing. His research interests mainly include crowd computing/crowdsourcing, social computing, and decision science.
Cite this article:   
Yi-Li Fang, Hai-Long Sun, Peng-Peng Chen, Ting Deng.Improving the Quality of Crowdsourced Image Labeling via Label Similarity[J]  Journal of Computer Science and Technology, 2017,V32(5): 877-889
URL:  
http://jcst.ict.ac.cn:8080/jcst/EN/10.1007/s11390-017-1770-7
Copyright 2010 by Journal of Computer Science and Technology