›› 2015, Vol. 30 ›› Issue (5): 998-1016.doi: 10.1007/s11390-015-1577-3

• Special Section on Software Systems • Previous Articles     Next Articles

CoreDevRec: Automatic Core Member Recommendation for Contribution Evaluation

Jing Jiang(蒋竞), Member, CCF, Jia-Huan He(贺佳欢), Xue-Yuan Chen(陈学渊)   

  1. State Key Laboratory of Software Development Environment, Beihang University, Beijing 100191, China
  • Received:2015-03-19 Revised:2015-07-16 Online:2015-09-05 Published:2015-09-05
  • About author:Jing Jiang received her B.S. and Ph.D. degrees in computer science from Peking University, Beijing, in 2007 and 2012, respectively. She is now an assistant professor in the State Key Laboratory of Software Development Environment of Beihang University, Beijing. Her research interests include software engineering, data mining, human factors and social aspects of software engineering.
  • Supported by:

    This work is supported by the National Natural Science Foundation of China under Grant No. 61300006 and the State Key Laboratory of Software Development Environment of China under Grant No. SKLSDE-2015ZX-24.

The pull-based software development helps developers make contributions flexibly and efficiently. Core members evaluate code changes submitted by contributors, and decide whether to merge these code changes into repositories or not. Ideally, code changes are assigned to core members and evaluated within a short time after their submission. However, in reality, some popular projects receive many pull requests, and core members have difficulties in choosing pull requests which are to be evaluated. Therefore, there is a growing need for automatic core member recommendation, which improves the evaluation process. In this paper, we investigate pull requests with manual assignment. Results show that 3.2%~40.6% of pull requests are manually assigned to specific core members. To assist with the manual assignment, we propose CoreDevRec to recommend core members for contribution evaluation in GitHub. CoreDevRec uses support vector machines to analyze different kinds of features, including file paths of modified codes, relationships between contributors and core members, and activeness of core members. We evaluate CoreDevRec on 18651 pull requests of five popular projects in GitHub. Results show that CoreDevRec achieves accuracy from 72.9% to 93.5% for top 3 recommendation. In comparison with a baseline approach, CoreDevRec improves the accuracy from 18.7% to 81.3% for top 3 recommendation. Moreover, CoreDevRec even has higher accuracy than manual assignment in the project TrinityCore. We believe that CoreDevRec can improve the assignment of pull requests.

[1] Gousios G, Pinzger M, van Deursen A. An exploratory study of the pull-based software development model. In Proc. the 36th ICSE, May 31-June 7, 2014, pp.345-355.

[2] Tsay J, Dabbish L, Herbsleb J. Influence of social and technical factors for evaluating contribution in GitHub. In Proc. the 36th ICSE, May 31-June 7, 2014, pp.356-366.

[3] Zhou M, Mockus A. What make long term contributors:Willingness and opportunity in OSS community. In Proc. the 34th ICSE, June 2012, pp.518-528.

[4] Yu Y, Wang H, Yin G, Ling C X. Who should review this pull-request:Reviewer recommendation to expedite crowd collaboration. In Proc. the 21st APSEC, December 2014, pp.335-342.

[5] Yu Y, Wang H, Yin G, Ling C X. Reviewer recommender of pull-requests in GitHub. In Proc. the 30th ICSME, September 29-October 3, 2014, pp.609-612.

[6] Rigby P C, Storey M A. Understanding broadcast based peer review on open source software projects. In Proc. the 33rd ICSE, May 2011, pp.541-550.

[7] Gousios G, Zaidman A, Storey M A, van Deursen A. Work practices and challenges in pull-based development:The integrator's perspective. In Proc. the 37th ICSE, May 2015.

[8] Thongtanunam P, Tantithamthavorn C, Kula R G, Yoshida N, Iida H, Matsumoto K. Who should review my code? A file location-based code-reviewer recommendation approach for modern code review. In Proc. the 22nd SANER, March 2015, pp.141-150.

[9] Dabbish L, Stuart C, Tsay J, Herbsleb J. Social coding in GitHub:Transparency and collaboration in an open software repository. In Proc. CSCW, February 2012, pp.1277-1286.

[10] Jiang J, Zhang L, Li L. Understanding project dissemination on a social coding site. In Proc. the 20th WCRE, October 2013, pp.132-141.

[11] Lehmann E L, D'Abrera H J M. Nonparametrics:Statistical Methods Based on Ranks. Prentice-Hall, 1998.

[12] Bird C, Pattison D, D'Souza R, Filkov V, Devanbu P. Latent social structure in open source projects. In Proc. the 16th SIGSOFT FSE, November 2008, pp.24-35.

[13] Xuan Q, Okano A, Devanbu P, Filkov V. Focus-shifting patterns of OSS developers and their congruence with call graphs. In Proc. the 22nd SIGSOFT FSE, November 2014, pp.401-412.

[14] Bird C, Gourley A, Devanbu P, Gertz M, Swaminathan A. Mining email social networks. In Proc. the 2006 MSR, May 2006, pp.137-143.

[15] Xuan Q, Filkov V. Building it together:Synchronous development in OSS. In Proc. the 36th ICSE, May 31-June 7, 2014, pp.222-233.

[16] Zhang W Q, Nie L M, Jiang H, Chen Z Y, Liu J. Developer social networks in software engineering:Construction, analysis, and applications. SCIENCE CHINA Information Sciences, 2014, 57(12):1-23.

[17] Crowston K, Wei K, Howison J, Wiggins A. Free/libre open source software development:What we know and what we do not know. ACM Computing Surveys, 2012, 44(2):7:1- 7:35.

[18] Bird C, Gourley A, Devanbu P, Swaminathan A, Hsu G. Open borders? Immigration in open source projects. In Proc. the 4th MSR, May 2007, Article No. 6.

[19] Boser B E, Guyon I M, Vapnik V N. A training algorithm for optimal margin classifiers. In Proc. the 5th Annual Workshop on Computational Learning Theory, July 1992, pp.144-152.

[20] Xia X, Lo D,Wang X, Yang X, Li S, Sun J. A comparative study of supervised learning algorithms for re-opened bug prediction. In Proc. the 17th CSMR, March 2013, pp.331-334.

[21] Park B, Bea J K. Using machine learning algorithms for housing price prediction:The case of Fairfax county, Virginia housing data. Expert Systems with Application, 2015, 42(6):2928-2934.

[22] Xia X, Lo D, Wang X, Zhou B. Accurate developer recommendation for bug resolution. In Proc. the 20th WCRE, October 2013, pp.72-81.

[23] Bettenburg N, Premraj R, Zimmermann T, Kim S. Duplicate bug reports considered harmful...Really? In Proc. the 24th ICSM, September 28-October 4, 2008, pp.337-345.

[24] Xuan J, Jiang H, Ren Z, Zou W. Developer prioritization in bug repositories. In Proc. the 34th ICSE, June 2012, pp.25-35.

[25] Balachandran V. Reducing human effort and improving quality in peer code reviews using automatic static analysis and reviewer recommendation. In Proc. the 35th ICSE, May 2013, pp.931-940.

[26] Kagdi H, Gethers M, Poshyvanyk D, Hammad M. Assigning change requests to software developers. Journal of Software:Evolution and Process, 2012, 24(1):3-33.

[27] Zweig M H, Campbell G. Receiver-operating characteristic (ROC) plots:A fundamental evaluation tool in clinical medicine. Clinical Chemistry, 1993, 39(4):561-577.

[28] Xuan Q, Fu C, Yu L. Ranking developer candidates by social links. Advances in Complex Systems, 2014, 7(17):1550005:1-1550005:19.

[29] Nurolahzade M, Nashehi S M, Khandkar S H, Rawal S. The role of patch review in software evolution:An analysis of the Mozilla Firefox. In Proc. IWPSE-Evol, August 2009, pp.9-17.

[30] Rigby P C, Germán DM, Storey M A. Open source software peer review practices:A case study of the Apache server. In Proc. the 30th ICSE, May 2008, pp.541-550.

[31] Bosu A, Carver J C. Peer code review in open source communities using review-board. In Proc. the 4th PLATEAU, October 2012, pp.17-24.

[32] Baysal O, Kononenko O, Holmes R, Godfrey M W. The influence of non-technical factors on code review. In Proc. the 20th WCRE, October 2013, pp.122-131.

[33] Linares-Vásquez M, Hossen K, Dang H, Kagdi H, Gethers M, Poshyvanyk D. Triaging incoming change requests:Bug or commit history, or code authorship? In Proc. the 28th ICSM, September 2012, pp.451-460.

[34] Jeong G, Kim S, Zimmermann T. Improving bug triage with bug tossing graphs. In Proc. the 17th ESEC/FSE, August 2009, pp.111-120.

[35] Matter D, Kuhn A, Nierstrasz O. Assigning bug reports using a vocabulary-based expertise model of developers. In Proc. the 6th MSR, May 2009, pp.131-140.

[36] Hossen M K, Kagdi H, Poshyvanyk D. Amalgamating source code authors, maintainers, and change proneness to triage change requests. In Proc. the 22nd ICPC, June 2014, pp.130-141.

[37] Hu H, Zhang H, Xuan J, Sun W. Effective bug triage based on historical bug-fix information. In Proc. the 25th ISSRE, November 2014, pp.122-132.

[38] Anvik J, Hiew L, Murphy G C. Who should fix this bug? In Proc. the 28th ICSE, May 2006, pp.361-370.

[39] Wu W, Zhang W, Yang Y, Wang Q. DREX:Developer recommendation with K-nearest-neighbor search and expertise ranking. In Proc. the 18th APSEC, December 2011, pp.389-396.

[40] Cubranic D, Murphy G C. Automatic bug triage using text categorization. In Proc. the 16th SEKE, June 2004, pp.92-97.

[41] Liu H, Ma Z, Shao W, Niu Z. Schedule of bad smell detection and resolution:A new way to save effort. IEEE Transactions on Software Engineering, 2012, 38(1):220- 235.

[42] Lee J B, Ihara A, Monden A, Matsumoto K. Patch reviewer recommendation in OSS projects. In Proc. the 20th APSEC, December 2013, pp.1-6.
No related articles found!
Full text



[1] Liu Mingye; Hong Enyu;. Some Covering Problems and Their Solutions in Automatic Logic Synthesis Systems[J]. , 1986, 1(2): 83 -92 .
[2] Chen Shihua;. On the Structure of (Weak) Inverses of an (Weakly) Invertible Finite Automaton[J]. , 1986, 1(3): 92 -100 .
[3] Gao Qingshi; Zhang Xiang; Yang Shufan; Chen Shuqing;. Vector Computer 757[J]. , 1986, 1(3): 1 -14 .
[4] Chen Zhaoxiong; Gao Qingshi;. A Substitution Based Model for the Implementation of PROLOG——The Design and Implementation of LPROLOG[J]. , 1986, 1(4): 17 -26 .
[5] Huang Heyan;. A Parallel Implementation Model of HPARLOG[J]. , 1986, 1(4): 27 -38 .
[6] Min Yinghua; Han Zhide;. A Built-in Test Pattern Generator[J]. , 1986, 1(4): 62 -74 .
[7] Tang Tonggao; Zhao Zhaokeng;. Stack Method in Program Semantics[J]. , 1987, 2(1): 51 -63 .
[8] Min Yinghua;. Easy Test Generation PLAs[J]. , 1987, 2(1): 72 -80 .
[9] Zhang Bo; Zhang Ling;. Statistical Heuristic Search[J]. , 1987, 2(1): 1 -11 .
[10] Zhu Hong;. Some Mathematical Properties of the Functional Programming Language FP[J]. , 1987, 2(3): 202 -216 .

ISSN 1000-9000(Print)

CN 11-2296/TP

Editorial Board
Author Guidelines
Journal of Computer Science and Technology
Institute of Computing Technology, Chinese Academy of Sciences
P.O. Box 2704, Beijing 100190 P.R. China
E-mail: jcst@ict.ac.cn
  Copyright ©2015 JCST, All Rights Reserved