We use cookies to improve your experience with our site.

Indexed in:

SCIE, EI, Scopus, INSPEC, DBLP, CSCD, etc.

Submission System
(Author / Reviewer / Editor)
Kai-Yuan Cai. Software Reliability Experimentation and Control[J]. Journal of Computer Science and Technology, 2006, 21(5): 697-707.
Citation: Kai-Yuan Cai. Software Reliability Experimentation and Control[J]. Journal of Computer Science and Technology, 2006, 21(5): 697-707.

Software Reliability Experimentation and Control

More Information
  • Received Date: April 01, 2006
  • Revised Date: July 09, 2006
  • Published Date: September 14, 2006
  • This paper classifies software researches as theoreticalresearches, experimental researches, and engineering researches, and ismainly concerned with the experimental researches with focus onsoftware reliability experimentation and control. The state-of-the-artof experimental or empirical studies is reviewed. A new experimentationmethodology is proposed, which is largely theory discovering oriented.Several unexpected results of experimental studies are presented tojustify the importance of software reliability experimentation andcontrol. Finally, a few topics that deserve future investigation areidentified.
  • \REF{[1]} Sipser M. Introduction to the Theory of Computation. PWS Publishing Company, 1997. \REF{[2]} M.H. Halstead. Elements of Software Science. North-Holland. 1977. \REF{[3]} Hudson G R. Program errors as a birth and death process. System Development Corporation. Report SP-3011, Santa Monica, CA, 1967. \REF{[4]} Basili V, Shull F, Lanubile F. Using experiments to build a body of knowledge. In {\it Proc. Ershov Memorial Conference}, 1999, pp.265--282. \REF{[5]} Jelinski Z, Moranda P B. Software Reliability Research. Statistical Computer Performance Evaluation, Greiberger W (ed.), Academic Press, 1972, pp.465--484. \REF{[6]} Musa J D. A theory of software reliability and its application. {\it IEEE Trans. Software Engineering}, 1975, SE-1(3): 312--327. \REF{[7]} Littlewood B, Verrall J. A Bayesian reliability growth model for computer software. {\it Applied Statistics}, 1973, 22(3): 332--346. \REF{[8]} Goel A L, Okumoto K. A time dependent error detection rate model for a large scale software system. In {\it Proc. the Third USA-Japan Computer Conference}, San Francisco, USA, 1978, pp.35--40. \REF{[9]} Nelson E C. Estimating software reliability from test data. {\it Microelectronics and Reliability}, 1978, 17(1): 67--74. \REF{[10]} Mills H D. On the statistical validation of computer program. FCS-72-6015, IBM Federal System Division, 1972. \REF{[11]} Avizienis A. Fault tolerant systems. {\it IEEE Trans. Computer}, 1976, C-25: 1304--1312. \REF{[12]} Randell B. System structure for software fault tolerance. IEEE {\it Trans. Software Engineering}, 1975, SE-1(2): 220--232. \REF{[13]} Cai K Y, Wen C Y, Zhang M.L. A critical review on software reliability modeling. {\it Reliability Engineering and System Safety}, 1991, 32: 357--371. \REF{[14]} Anderson T, Barret P A, Halliwell D N, Moulding M R. Software fault tolerance: An evaluation. {\it IEEE Trans. Software Engineering}, 1985, SE-11(12): 1502--1510. \REF{[15]} Knight J C, Leveson N G. An experimental evaluation of the assumption of independence in multi-version programming. {\it IEEE Trans. Software Engineering}, 1986, SE-12(1): 96--109. \REF{[16]} Butler R W, Finelli G B. The infeasibility of quantifying the reliability of life-critical real-time software. {\it IEEE Trans. Software Engineering}, 1993, 19(1): 3--12. \REF{[17]} Littlewood B, Strigini L, Validation of ultra-high dependabi\-lity for software-based systems. {\it Communications of the ACM}, 1993, 36(11): 69--80. \REF{[18]} Reussner R H, Schmidt H W, Poernomo I H. Reliability prediction for component-based architectures. {\it Journal of Systems and Software}, 2003, 66: 241--252. \REF{[19]} Huynh T, Miller J. Further investigations into evaluating website reliability. In {\it Proc. Int. Symp. Empirical Software Engineering}, Noosa Heads, Australia, 2005, pp.162--171. \REF{[20]} Xie M. Software Reliability Modeling. World Scientific, 1991. \REF{[21]} Cai K Y. Software Defect and Operational Profile Modeling. Academic Publishers, 1998. \REF{[22]} Cai K Y, Wang X Y. Towards a control-theoretical approach to software fault-tolerance. In {\it Proc. the 4th Int. Conference on Quality Software}, IEEE Computer Society Press, Braunschweig, Germany, 2004, pp.198--205. \REF{[23]} Pretschner A {\it et al}. One evaluation of model-based testing and its automation. In {\it Proc. Int. Conf. Software Engineering}, St. Louis, MO, USA, 2005, pp.302--401. \REF{[24]} E M Clarke Jr, O Grumberg, D A Peled. Model Checking. The MIT Press, 1999. \REF{[25]} Biffl B, Gutjahr W J. Using a reliability growth model to control software inspection. \it Empirical Software Engineering, \rm 2002, 7: 257--284. \REF{[26]} Stringfellow C, Andrews A A. An empirical method for selecting software reliability growth models. \it Empirical Software Engineering, \rm 2002, 7: 319--343. \REF{[27]} Henningsson K, Wohlin C. Assuring fault classification agreement---An empirical evaluation. In \it Proc. Int. Symp. Empirical Software Engineering, \rm 2004, pp.95--104. \REF{[28]} Curtis B. Measurement and experimentation in software engineering. \it Proc. the IEEE, \rm 1980, 68(9): 1144--1157. \REF{[29]} Tichy W F, Lukowicz P, Prechelt L \it et al. \rm Experimental evaluation in computer science: A quantitative study. \it Journal of Systems and Software, \rm 1995, 28: 9--18. \REF{[30]} Tichy W F. Should computer scientists experiment more? \it Computer, \rm May 1998, 31(5): 32--40. \REF{[31]} Perry D E, Porter A A, Votta L G. Empirical studies of software engineering: A roadmap. In \it Proc. ICSE Workshop on Future of Software Engineering, \rm Edinburgh, UK, 2000, pp.347--355. \REF{[32]} Harrison R, Badoo N, Barry E \it et al. \rm Directions and methodologies for empirical software engineering research. \it Empirical Software Engineering, \rm 1999, 4(4): 405--410. \REF{[33]} Sjoberg D I K, Hannay J E, Hansen O \it et al. \rm A survey of controlled experiments in software engineering. \it IEEE Trans. Software Engineering, \rm 2005, 31(9): 733--753. \REF{[34]} Fenton N, Pfleeger S L, Glass R L. Science and substance: A challenge to software engineering. \it IEEE Software, \rm July 1994, pp.88--95. \REF{[35]} Zelkowitz M V, Wallance D R. Experimental models for validating technology. \it IEEE Software, \rm May 1998, pp.23--31. \REF{[36]} Rothermel G, Elbaum S, Malishevsky A G \it et al. \rm On test suite composition and cost-effective regression testing. \it ACM Trans. Software Engineering and Methodology, \rm 2004, 13(3): 277--331. \REF{[37]} Juristo N, Moreno A M, Vegas S. Reviewing 25 years of testing technique experiments. \it Empirical Software Engineering, \rm 2004, 9: 7--44. \REF{[38]} Zendler A. A preliminary software engineering theory as investigated by published experiments. \it Empirical Software Engineering, \rm 2001, 6: 161--180. \REF{[39]} IEEE. IEEE Standard Glossary of Software Engineering Terminology. IEEE Std 610.12-1990, 1990. \REF{[40]} Kitchenham B A, Pfleeger S L, Pickard L M \it et al. \rm Preliminary guidelines for empirical research in software engineering. \it IEEE Trans. Software Engineering, \rm 2002, 28(8): 721--734. \REF{[41]} Cater-Steel A, Toleman M, Rout T. Addressing the challenges of surveys in software engineering. In \it Proc. 2005 Int. Symp. Empirical Software Engineering, \rm 2005, pp.204--213. \REF{[42]} Jedlitschka A, Pfahl D. Reporting guidelines for controlled experiments in software engineering. In \it Proc. 2005 Int. Symp. Empirical Software Engineering, \rm Noosa Heads, Australia, 2005, pp.95--104. \REF{[43]} Do H, Elbaum S, Rothermel G. Infrastructure support for controlled experimentation with software testing and regression testing techniques. In \it Proc. 2004 Int. Symp. Empirical Software Engineering, \rm Redondo Beach, CA, USA, 2004, pp.60--70. \REF{[44]} Boehm B \it et al. \rm Using empirical testbeds to accelerate technology maturity and transition: The SCRover experience. In \it Proc. 2004 Int. Symp. Empirical Software Engineering, \rm Redondo Beach, CA, USA, 2004, pp.117--126. \REF{[45]} Cai K Y, Chen L. Analyzing software science data with partial repeatability. \it Journal of Systems and Software, \rm 2002, 63: 173--186. \REF{[46]} Xie M. Software Reliability Modeling. World Scientific, 1991. \REF{[47]} Cai K Y. Software Defect and Operational Profile Modeling. Kluwer Academic Publishers, 1998. \REF{[48]} Pham H. Software reliability and cost models: Perspectives, comparison, and practice. \it European Journal of Operational Research, \rm 2003, 149: 475--489. \REF{[49]} Vokolos F I, Frankl R H. Empirical evaluation of the textual differencing regression testing technique. In \it Proc. the Int. Conf. Software Maintenance, \rm Bethesda, Maryland, USA, November 1998, pp.44--53. \REF{[50]} Rothermel G, Untch R H, Chu C \it et al. \rm Prioritizing test cases for regression testing. \it IEEE Trans. Software Engineering, \rm 2001, 27(10): 929--948. \REF{[51]} Grassi V, Mirandola R. Devivation of Markov models for effectiveness analysis of adaptable software architectures for mobile computing. \it IEEE Trans. Mobile Computing, \rm 2003, 4(2): 114--131. \REF{[52]} Goseva-Popstojanova K, Trivedi K. Architecture-based approach to reliability assessment of software systems. \it Performance Evaluation, \rm 2001, 45: 179--204. \REF{[53]} Cai K Y, Lei X F, Shi Y \it et al. \rm Partial-repeatability as a new form of uncertainty. Fuzzy Logic, Software Computing and Computational Intelligence, Yingming Liu, Guoqing Chen, Mingsheng Ying (eds.), \it Proc. 11th Int. Fuzzy Systems Association World Congress, \rm Tsinghua University Press/Springer, Beijing, 2005, pp.157--166. \REF{[54]} Whittaker J A. What is software testing? And why is it so hard? \it IEEE Software, \rm January/February, 2000, 17(1): 70--79. \REF{[55]} Cai K Y. Optimal software testing and adaptive software testing in the context of software cybernetics. \it Information and Software Technology, \rm 2002, 44: 841--855. \REF{[56]} Cai K Y, Li Y C, Liu K. Optimal and adaptive testing for software reliability assessment. \it Information and Software Technology, \rm 2004, 46: 989--1000. \REF{[57]} Guo Y L, Shen F J. The History of Physics. Tsinghua University Press, 1993. (in Chinese) \REF{[58]} Johannes Kepler. His life, his laws and times. http://kepler.nasa.gov/johannes/. \REF{[59]} Li M, Vitanyi P M B. An Introduction to Kolmogorov Complexity and Its Applications, Second Edition. Springer-Verlag, 1997. \REF{[60]} Wiener N. Cybernetics: Or Control and Communication in the Animal and the Machine. John Wiley \& Sons, 1948. \REF{[61]} Lyu M R (ed.). Handbook of Software Reliability Engineering. McGraw-Hill, 1996. \REF{[62]} Fenton N, Littlewood B, Neil M. Applying Bayesian belief network in systems dependability assessment. In \it Proc. Safety Critical Systems Symposium, \rm Springer-Verlag, Leeds, UK, 1996, pp.71--94. \REF{[63]} Valiant L G. A theory of the learnable. \it Communications of the ACM, \rm 1984, 27(11): 1134--1142. \REF{[64]} Angluim D. Queries revisited. \it Theoretical Computer Science, \rm 2004, 313: 175--194. \REF{[65]} C de la Higuera. A bibliographical study of grammatical inference. \it Pattern Recognition, \rm 2005, 38: 1332--1348. \REF{[66]} Cai K Y, Cangussu J W, DeCarlo R A \it et al. \rm An overview of software cybernetics. In \it Proc. the 11th Int. Workshop on Software Technology and Engineering Practice, \rm IEEE Computer Society Press, Amsterdam, Holland, 2003, pp.77--86. \REF{[67]} Belli F, Cai K Y, DeCarlo R A \it et al. \rm Introduction to the special section on software cybernetics. \it Journal of Systems and Software, \rm article in press, available online, 2006.
  • Related Articles

    [1]WEN Jirong, CHEN Hong, WANG Shan. POTENTIAL: A Highly Adaptive Core of Parallel Database System[J]. Journal of Computer Science and Technology, 2000, 15(6): 527-541.
    [2]LI Xiaowei, Paul Y.S. Cheung. Exploiting Deterministic TPG for Path Delay Testing[J]. Journal of Computer Science and Technology, 2000, 15(5): 472-479.
    [3]LING Yu, LI Shu, ZHANG Hui, HAN Chengde. Timing-Sequence Testing of Parallel Programs[J]. Journal of Computer Science and Technology, 2000, 15(1): 84-95.
    [4]LU Sanglu, ZHOU Xiaoboand, XIE Li. A Model for Dynamic Adaptive Coscheduling[J]. Journal of Computer Science and Technology, 1999, 14(3): 267-275.
    [5]XU Mingwei, WU Jianping. A Formal Approach to Protocol Performance Testing[J]. Journal of Computer Science and Technology, 1999, 14(1): 81-87.
    [6]David de Frutos-Escrig, Luis Liana-Diaz, Manuel Nunez. An invitation to Friendly Testing[J]. Journal of Computer Science and Technology, 1998, 13(6): 531-545.
    [7]Hao Ruibing, Wu Jianping. A Formal Approach to Protocol Interoperability Testing[J]. Journal of Computer Science and Technology, 1998, 13(1): 79-90.
    [8]Sun Yuning, Wang Xiaoming, Shi Wanchun. ICTSSE: An Object-Oriented IC Test Software Supporting Environment[J]. Journal of Computer Science and Technology, 1995, 10(5): 447-454.
    [9]Zhou Jianqiang, Xie Li, Dai Fei, Sun Zhongxiu. Adaptive Memory Coherence Algorithms in DSVM[J]. Journal of Computer Science and Technology, 1994, 9(4): 365-372.
    [10]Zhou Jianqiang, Xie Li, Sun Zhongxiu, Zhu Genjiang. An Adaptive Strategy Integrating Locking with Optimistic Concurrency Control[J]. Journal of Computer Science and Technology, 1993, 8(4): 61-69.

Catalog

    Article views (24) PDF downloads (1922) Cited by()
    Related

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return