? 基于串谋检测的众包结果推理方法
Journal of Computer Science and Technology
Quick Search in JCST
 Advanced Search 
      Home | PrePrint | SiteMap | Contact Us | Help
 
Indexed by   SCIE, EI ...
Bimonthly    Since 1986
Journal of Computer Science and Technology 2018, Vol. 33 Issue (2) :351-365    DOI: 10.1007/s11390-018-1823-6
Data Management and Data Mining << Previous Articles | Next Articles >>
基于串谋检测的众包结果推理方法
Peng-Peng Chen1,2, Student Member, CCF, ACM, Hai-Long Sun1,2*, Member, CCF, ACM, IEEE, Yi-Li Fang1,2*, Member, CCF, ACM, Jin-Peng Huai1,2, Fellow, CCF, Member, ACM, IEEE
1 State Key Laboratory of Software Development Environment, School of Computer Science and Engineering Beihang University, Beijing 100191, China;
2 Beijing Advanced Innovation Center for Big Data and Brain Computing, Beijing 100191, China
Collusion-Proof Result Inference in Crowdsourcing
Peng-Peng Chen1,2, Student Member, CCF, ACM, Hai-Long Sun1,2*, Member, CCF, ACM, IEEE, Yi-Li Fang1,2*, Member, CCF, ACM, Jin-Peng Huai1,2, Fellow, CCF, Member, ACM, IEEE
1 State Key Laboratory of Software Development Environment, School of Computer Science and Engineering Beihang University, Beijing 100191, China;
2 Beijing Advanced Innovation Center for Big Data and Brain Computing, Beijing 100191, China

摘要
参考文献
相关文章
Download: [PDF 735KB]  
摘要 在众包中,通常考虑工人独立处理任务并且提交答案,从而确保答案的多样性。事实上,当前研究表明来自通用平台的工人之间存在隐式的协作关系。工人为了付出少量的劳动获取更多的报酬,可能提供重复的答案进行串谋。该种行为会严重损害最终众包结果的质量。然而存在的众包方法均没有考虑到串谋对众包结果推理的影响。因此本文提出一种基于串谋检测的众包结果推理方法。利用工人表现的变化率,通过计算删除重复答案之前和之后工人平均表现的差异来检测串谋行为产生的重复答案,并考虑入结果推理方法中,以确保汇聚结果的质量。基于众包平台的真实数据和仿真数据进行了大量的实验评估。实验结果表明了本文方法的优越性。
关键词众包   质量控制   串谋   协作众包   结果汇聚     
Abstract: In traditional crowdsourcing, workers are expected to provide independent answers to tasks so as to ensure the diversity of answers. However, recent studies show that the crowd is not a collection of independent workers, but instead that workers communicate and collaborate with each other. To pursue more rewards with little effort, some workers may collude to provide repeated answers, which will damage the quality of the aggregated results. Nonetheless, there are few efforts considering the negative impact of collusion on result inference in crowdsourcing. In this paper, we are specially concerned with the Collusion-Proof result inference problem for general crowdsourcing tasks in public platforms. To that end, we design a metric, the worker performance change rate, to identify the colluded answers by computing the difference of the mean worker performance before and after removing the repeated answers. Then we incorporate the collusion detection result into existing result inference methods to guarantee the quality of the aggregated results even with the occurrence of collusion behaviors. With real-world and synthetic datasets, we conducted an extensive set of evaluations of our approach. The experimental results demonstrate the superiority of our approach in comparison with the state-of-the-art methods.
Keywordscrowdsourcing   quality control   collusion   collaborative crowdsourcing   result inference     
Received 2017-04-17;
本文基金:

This work was supported partly by the National Basic Research 973 Program of China under Grant Nos. 2015CB358700 and 2014CB340304, the National Natural Science Foundation of China under Grant No. 61421003, and the Open Fund of the State Key Laboratory of Software Development Environment under Grant No. SKLSDE-2017ZX-14.

通讯作者: Hai-Long Sun     Email: sunhl@buaa.edu.cn
About author: Peng-Peng Chen is a Ph.D. student in the School of Computer Science and Engineering, Beihang University, Beijing. His research interests mainly include crowd computing/crowdsourcing, and social computing. He is a student member of CCF and ACM
引用本文:   
Peng-Peng Chen, Hai-Long Sun, Yi-Li Fang, Jin-Peng Huai.基于串谋检测的众包结果推理方法[J]  Journal of Computer Science and Technology , 2018,V33(2): 351-365
Peng-Peng Chen, Hai-Long Sun, Yi-Li Fang, Jin-Peng Huai.Collusion-Proof Result Inference in Crowdsourcing[J]  Journal of Computer Science and Technology, 2018,V33(2): 351-365
链接本文:  
http://jcst.ict.ac.cn:8080/jcst/CN/10.1007/s11390-018-1823-6
Copyright 2010 by Journal of Computer Science and Technology