• Articles • Previous Articles     Next Articles

Improving Fault Detection in Modified Code --- A Study from the Telecommunication Industry

Piotr Tomaszewski, Lars Lundberg, and Haa kan Grahn   

  1. School of Engineering, Blekinge Institute of Technology, SE-372 25 Ronneby, Sweden
  • Received:2006-03-15 Revised:2007-02-15 Online:2007-05-10 Published:2007-05-10

Many software systems are developed in a number of consecutive releases. In each release not only new code is added but also existing code is often modified. In this study we show that the modified code can be an important source of faults. Faults are widely recognized as one of the major cost drivers in software projects. Therefore, we look for methods that improve the fault detection in the modified code. We propose and evaluate a number of prediction models that increase the efficiency of fault detection. To build and evaluate our models we use data collected from two large telecommunication systems produced by Ericsson. We evaluate the performance of our models by applying them both to a different release of the system than the one they are built on and to a different system. The performance of our models is compared to the performance of the theoretical best model, a simple model based on size, as well as to analyzing the code in a random order (not using any model). We find that the use of our models provides a significant improvement over not using any model at all and over using a simple model based on the class size. The gain offered by our models corresponds to 38~57% of the theoretical maximum gain.

Key words: Active Message; user-level communication; software overhead; interconnection network; Dawning-1000;

[1]Sommerville I. Software Engineering. Boston: Addison-Wesley, 2004.

[2] Cartwright M, Shepperd M. An empirical investigation of an object-oriented software system. -\it IEEE Trans. Software Engineering}, 2000, 26(8): 786$\sim$796.

[3] Boehm B, Basili V R. Software defect reduction top 10 list. -\it Computer}, 2001, 34: 135$\sim$137.

[4] Fenton N, Ohlsson N. Quantitative analysis of faults and failures in a complex software system. -\it IEEE Trans. Software Engineering}, 2000, 26(8): 797$\sim$814.

[5] Ohlsson N, Eriksson A C, Helander M. Early risk-management by identification of fault-prone modules. -\it Empirical Software Engineering}, 1997, 2(2): 166$\sim$173.

[6] Briand L C, Wust J, Daly J W, Porter D W. Exploring the relationship between design measures and software quality in object-oriented systems. -\it The Journal of Systems and Software}, 2000, 51(3): 245$\sim$273.

[7] Briand L C, Wust J, Ikonomovski S V \it et al. \rm Investigating quality factors in object-oriented designs: An industrial case study. In -\it Proc. the 1999 Int. Conf. Software Eng.}, Los Angeles, USA, 1999, pp.345$\sim$354.

[8] K El Emam, W L Melo, J C Machado. The prediction of faulty classes using object-oriented design metrics. -\it The Journal of Systems and Software}, 2001, 56(1): 63$\sim$75.

[9] Tomaszewski P, Lundberg L, Grahn H. Increasing the efficiency of fault detection in modified code. In -\it Proc. Asia Pacific Software Engineering Conference}, Taipei, 2005, pp.421$\sim$430.

[10] Zhao M, Wohlin C, Ohlsson N, Xie M. A comparison between software design and code metrics for the prediction of software fault content. -\it Information and Software Technology}, 1998, 40(14): 801$\sim$809.

[11] Pighin M, Marzona A. An empirical analysis of fault persistence through software releases. In -\it Proc. the Int. Symp. Empirical Software Engineering}, Rome, Italy, 2003, pp.206$\sim$212.

[12] Pighin M, Marzona A. Reducing corrective maintenance effort considering module's history. In -\it Proc. Ninth European Conference on Software Maintenance and Reengineering}, Manchester, UK, 2005, pp.232$\sim$235.

[13] Selby R W. Empirically based analysis of failures in software systems. -\it IEEE Trans. Reliability}, 1990, 39(4): 444$\sim$454.

[14] Gascoyne S. Productivity improvements in software testing with test automation. -\it Electronic Engineering}, 2000, 72(885): 65$\sim$67.

[15] A Gunes Koru, J Tian. An empirical comparison and characterization of high defect and high complexity modules. -\it Journal of Systems and Software}, 2003, 67(3): 153$\sim$163.

[16] Khoshgoftaar T M, Allen E B, Deng J. Controlling overfitting in software quality models: Experiments with regression trees and classification. In -\it Proc. The 7th Int. Software Metrics Symposium}, London, UK, 2001, pp.190$\sim$198.

[17] Khoshgoftaar T M, Allen E B, Jianyu D. Using regression trees to classify fault-prone software modules. -\it IEEE Trans. Reliability}, 2002, 51(4): 455$\sim$462.

[18] Khoshgoftaar T M, Allen E B, Jones W D, Hudepohl J P. Accuracy of software quality models over multiple releases. -\it Annals of Software Engineering}, 2000, 9(1-4): 103$\sim$116.

[19] Khoshgoftaar T M, Seliya N. Fault prediction modeling for software quality estimation: Comparing commonly used techniques. -\it Empirical Software Engineering}, 2003, 8(3): 255$\sim$283.

[20] Ohlsson N, Zhao M, Helander M. Application of multivariate analysis for software fault prediction. -\it Software Quality Journal}, 1998, 7(1): 51$\sim$66.

[21] Tomaszewski P, H\aa kansson J, Lundberg L, Grahn H. The accuracy of fault prediction in modified code --Statistical model vs. expert estimation. In -\it Proc. 13th Annual IEEE Int. Conf. and Workshop on the Engineering of Computer Based Systems}, Potsdam, Germany, 2006, pp.334$\sim$343.

[22] Chidamber S R, Darcy D P, Kemerer C F. Managerial use of metrics for object-oriented software: An exploratory analysis. -\it IEEE Trans. Software Engineering}, 1998, 24(8): 629$\sim$639.

[23] Nikora A P, Munson J C. Developing fault predictors for evolving software systems. In -\it Proc. The Ninth Int. Software Metrics Symposium}, Sydney, Australia, 2003, pp.338$\sim$349.

[24] Ostrand T J, Weyuker E J, Bell R M. Predicting the location and number of faults in large software systems. -\it IEEE Trans. Software Engineering}, 2005, 31(4): 340$\sim$355.

[25] Yu P, Systa T, Muller H. Predicting fault-proneness using OO metrics. An industrial case study. In -\it Proc. The Sixth European Conference on Software Maintenance and Reengineering}, Budapest, Hungary, 2002, pp.99$\sim$107.

[26] Chidamber S R, Kemerer C F. A metrics suite for object oriented design. -\it IEEE Trans. Software Engineering}, 1994, 20(6): 476$\sim$494.

[27] Fioravanti F, Nesi P. A study on fault-proneness detection of object-oriented systems. In -\it Proc. Fifth European Conf. Software Maintenance and Reengineering}, Lisbon, Portugal, 2001, pp.121$\sim$130.

[28] Fenton N, Neil M. A critique of software defect prediction models. -\it IEEE Trans. Software Engineering}, 1999, 25(5): 675$\sim$689.

[29] Keppel G. Design and Analysis: A Researcher's Handbook. Upper Saddle River: Prentice Hall, NJ, 2004.

[30] Milton J S, Arnold J C. Introduction to Probability and Statistics: Principles and Applications for Engineering and the Computing Sciences. Boston: McGraw-Hill, 2003.

[31] Nagappan N, Ball T. Use of relative code churn measures to predict system defect density. In -\it Proc. the 27th Int. Conf. Software Engineering}, St. Louis, USA, 2005, pp.284$\sim$292.

[32] Munson J C, Elbaum S G. Code churn: A measure for estimating the impact of code change. In -\it Proc. the Int. Conf. Software Maintenance}, Bethesda, USA, 1998, pp.24$\sim$31.

[33] Graham I. Migrating to Object Technology. Wokingham, England; Reading, Mass.: Addison-Wesley Pub. Co., 1995.

[34] Henderson-Sellers B, Constantine L L, Graham I M. Coupling and cohesion (towards a valid metrics suite for object-oriented analysis and design). -\it Object Oriented Systems}, 1996, 3(3): 143$\sim$158.

[35] Scientific Toolworks Inc. Understand C++ Homepage. Last accessed on 2006-03-13.

[36] LOCC Project Homepage. The Collaborative Software Development Laboratory. University of Hawaii, USA, last accessed on 2005-11-10, http://csdl.ics.hawaii.edu/Tools/LOCC/.

[37] Kitchenham B, Pickard L, Pfleeger S L. Case studies for method and tool evaluation. -\it IEEE Software}, 1995, 12(4): 52$\sim$62.

[38] Kitchenham B A, Pfleeger S L, Pickard L M \it et al. \rm Preliminary guidelines for empirical research in software engineering. -\it IEEE Trans. Software Engineering}, 2002, 28(8): 721.

[39] Wohlin C, Runeson P, H\"ost M \it et al. \rm Experimentation in Software Engineering: An Introduction. Boston: Kluwer, 2000.

[40] Rees D G. Essential Statistics, London; New York: Chapman \& Hall, 1995.

[41] SPSS Inc., SPSS Package Homepage, http://www.spss.com/, last accessed on 2006-03-13.

[42] Aczel A D, Sounderpandian J. Complete Business Statistics. Boston: McGraw-Hill, 2006.

[43] Host M, Regnell B, Wohlin C. Using students as subjects---A comparative study of students and professionals in lead-time impact assessment. -\it Empirical Software Engineering}, 2000, 5(3): 201$\sim$214.
[1] Wei-Bei Fan, Jian-Xi Fan, Cheng-Kuan Lin, Yan Wang, Yue-Juan Han, Ru-Chuan Wang. Optimally Embedding 3-Ary n-Cubes into Grids [J]. Journal of Computer Science and Technology, 2019, 34(2): 372-387.
[2] Xi Wang, Jian-Xi Fan, Cheng-Kuan Lin, Jing-Ya Zhou, Zhao Liu. BCDC: A High-Performance, Server-Centric Data Center Network [J]. , 2018, 33(2): 400-416.
[3] Dong-Fang Zhou, Jian-Xi Fan, Cheng-Kuan Lin, Bao-Lei Cheng, Jing-Ya Zhou, Zhao Liu. Optimal Path Embedding in the Exchanged Crossed Cube [J]. , 2017, 32(3): 618-629.
[4] Wen-Tao Bao, Bin-Zhang Fu, Ming-Yu Chen, Li-Xin Zhang. A High-Performance and Cost-Effcient Interconnection Network for High-Density Servers [J]. , 2014, 29(2): 281-292.
[5] Tiziana Calamoneri, Saverio Caminiti, and Rossella Petreschi. A General Approach to {\it\bfseries L$($h,k$)$}-Label Interconnection Networks [J]. , 2008, 23(4 ): 652-659 .
[6] Shu-Ming Zhou and Wen-Jun Xiao. A New Family of Interconnection Networks of Fixed Degree Three [J]. , 2004, 19(2): 0-0.
[7] Gao-Cai Wang, Jian-Er Chen, and Guo-Jun Wang. On Fault Tolerance of 3-Dimensional Mesh Networks [J]. , 2004, 19(2): 0-0.
[8] MA Jie(马捷),HE Jin(贺劲),MENG Dan(孟丹)and LI Guojie(李国杰). BCL-3:A High Performance Basic Communication Protocol for Commodity Superserver DAWNING-3000 [J]. , 2001, 16(6): 0-0.
[9] WANG Xiaodong; XU Ming; ZHOU Xingming;. Fast Multicast on Multistage Interconnection Networks Using Multi-Head Worms [J]. , 1999, 14(3): 250-258.
[10] WANG Deqiang; ZHAO Lianchang;. The Twisted-Cube Connected Networks [J]. , 1999, 14(2): 181-187.
[11] XIAO Limin; ZHU Mingfa;. Exploiting the Capabilities of the Interconnection Network on Dawning-1000 [J]. , 1999, 14(1): 49-55.
[12] Chi Xuebin;. Parallel Implementation of Linear Algebra Problems on Dawning-1000 [J]. , 1998, 13(2): 141-146.
[13] Sun Ninghui; Liu Wenzhuo; Liu Hong; Wang Chuanbao; Lu Xuelin; Zhang Hao;. Dawning-1000 PROOS Distributed Operating System [J]. , 1997, 12(2): 160-166.
[14] Zhou Yingquan; Min Yinghua;. A Kind of Multistage Interconnection Networks with Multiple Paths [J]. , 1996, 11(4): 395-404.
[15] Liao Xianzhi; Jin Lan;. Rendezvous Facilities in a Distributed Computer System [J]. , 1995, 10(2): 188-192.
Full text



[1] Liu Mingye; Hong Enyu;. Some Covering Problems and Their Solutions in Automatic Logic Synthesis Systems[J]. , 1986, 1(2): 83 -92 .
[2] Chen Shihua;. On the Structure of (Weak) Inverses of an (Weakly) Invertible Finite Automaton[J]. , 1986, 1(3): 92 -100 .
[3] Gao Qingshi; Zhang Xiang; Yang Shufan; Chen Shuqing;. Vector Computer 757[J]. , 1986, 1(3): 1 -14 .
[4] Chen Zhaoxiong; Gao Qingshi;. A Substitution Based Model for the Implementation of PROLOG——The Design and Implementation of LPROLOG[J]. , 1986, 1(4): 17 -26 .
[5] Huang Heyan;. A Parallel Implementation Model of HPARLOG[J]. , 1986, 1(4): 27 -38 .
[6] Shen Li; Stephen Y.H.Su;. Generalized Parallel Signature Analyzers with External Exclusive-OR Gates[J]. , 1986, 1(4): 49 -61 .
[7] Min Yinghua; Han Zhide;. A Built-in Test Pattern Generator[J]. , 1986, 1(4): 62 -74 .
[8] Gong Zhenhe;. On Conceptual Model Specification and Verification[J]. , 1987, 2(1): 35 -50 .
[9] Tang Tonggao; Zhao Zhaokeng;. Stack Method in Program Semantics[J]. , 1987, 2(1): 51 -63 .
[10] Min Yinghua;. Easy Test Generation PLAs[J]. , 1987, 2(1): 72 -80 .

ISSN 1000-9000(Print)

CN 11-2296/TP

Editorial Board
Author Guidelines
Journal of Computer Science and Technology
Institute of Computing Technology, Chinese Academy of Sciences
P.O. Box 2704, Beijing 100190 P.R. China
E-mail: jcst@ict.ac.cn
  Copyright ©2015 JCST, All Rights Reserved