Software Reliability Experimentation and Control

\REF{[1]} Sipser M. Introduction to the Theory of Computation. PWS Publishing Company, 1997. \REF{[2]} M.H. Halstead. Elements of Software Science. North-Holland. 1977. \REF{[3]} Hudson G R. Program errors as a birth and death process. System Development Corporation. Report SP-3011, Santa Monica, CA, 1967. \REF{[4]} Basili V, Shull F, Lanubile F. Using experiments to build a body of knowledge. In {\it Proc. Ershov Memorial Conference}, 1999, pp.265--282. \REF{[5]} Jelinski Z, Moranda P B. Software Reliability Research. Statistical Computer Performance Evaluation, Greiberger W (ed.), Academic Press, 1972, pp.465--484. \REF{[6]} Musa J D. A theory of software reliability and its application. {\it IEEE Trans. Software Engineering}, 1975, SE-1(3): 312--327. \REF{[7]} Littlewood B, Verrall J. A Bayesian reliability growth model for computer software. {\it Applied Statistics}, 1973, 22(3): 332--346. \REF{[8]} Goel A L, Okumoto K. A time dependent error detection rate model for a large scale software system. In {\it Proc. the Third USA-Japan Computer Conference}, San Francisco, USA, 1978, pp.35--40. \REF{[9]} Nelson E C. Estimating software reliability from test data. {\it Microelectronics and Reliability}, 1978, 17(1): 67--74. \REF{[10]} Mills H D. On the statistical validation of computer program. FCS-72-6015, IBM Federal System Division, 1972. \REF{[11]} Avizienis A. Fault tolerant systems. {\it IEEE Trans. Computer}, 1976, C-25: 1304--1312. \REF{[12]} Randell B. System structure for software fault tolerance. IEEE {\it Trans. Software Engineering}, 1975, SE-1(2): 220--232. \REF{[13]} Cai K Y, Wen C Y, Zhang M.L. A critical review on software reliability modeling. {\it Reliability Engineering and System Safety}, 1991, 32: 357--371. \REF{[14]} Anderson T, Barret P A, Halliwell D N, Moulding M R. Software fault tolerance: An evaluation. {\it IEEE Trans. Software Engineering}, 1985, SE-11(12): 1502--1510. \REF{[15]} Knight J C, Leveson N G. An experimental evaluation of the assumption of independence in multi-version programming. {\it IEEE Trans. Software Engineering}, 1986, SE-12(1): 96--109. \REF{[16]} Butler R W, Finelli G B. The infeasibility of quantifying the reliability of life-critical real-time software. {\it IEEE Trans. Software Engineering}, 1993, 19(1): 3--12. \REF{[17]} Littlewood B, Strigini L, Validation of ultra-high dependabi\-lity for software-based systems. {\it Communications of the ACM}, 1993, 36(11): 69--80. \REF{[18]} Reussner R H, Schmidt H W, Poernomo I H. Reliability prediction for component-based architectures. {\it Journal of Systems and Software}, 2003, 66: 241--252. \REF{[19]} Huynh T, Miller J. Further investigations into evaluating website reliability. In {\it Proc. Int. Symp. Empirical Software Engineering}, Noosa Heads, Australia, 2005, pp.162--171. \REF{[20]} Xie M. Software Reliability Modeling. World Scientific, 1991. \REF{[21]} Cai K Y. Software Defect and Operational Profile Modeling. Academic Publishers, 1998. \REF{[22]} Cai K Y, Wang X Y. Towards a control-theoretical approach to software fault-tolerance. In {\it Proc. the 4th Int. Conference on Quality Software}, IEEE Computer Society Press, Braunschweig, Germany, 2004, pp.198--205. \REF{[23]} Pretschner A {\it et al}. One evaluation of model-based testing and its automation. In {\it Proc. Int. Conf. Software Engineering}, St. Louis, MO, USA, 2005, pp.302--401. \REF{[24]} E M Clarke Jr, O Grumberg, D A Peled. Model Checking. The MIT Press, 1999. \REF{[25]} Biffl B, Gutjahr W J. Using a reliability growth model to control software inspection. \it Empirical Software Engineering, \rm 2002, 7: 257--284. \REF{[26]} Stringfellow C, Andrews A A. An empirical method for selecting software reliability growth models. \it Empirical Software Engineering, \rm 2002, 7: 319--343. \REF{[27]} Henningsson K, Wohlin C. Assuring fault classification agreement---An empirical evaluation. In \it Proc. Int. Symp. Empirical Software Engineering, \rm 2004, pp.95--104. \REF{[28]} Curtis B. Measurement and experimentation in software engineering. \it Proc. the IEEE, \rm 1980, 68(9): 1144--1157. \REF{[29]} Tichy W F, Lukowicz P, Prechelt L \it et al. \rm Experimental evaluation in computer science: A quantitative study. \it Journal of Systems and Software, \rm 1995, 28: 9--18. \REF{[30]} Tichy W F. Should computer scientists experiment more? \it Computer, \rm May 1998, 31(5): 32--40. \REF{[31]} Perry D E, Porter A A, Votta L G. Empirical studies of software engineering: A roadmap. In \it Proc. ICSE Workshop on Future of Software Engineering, \rm Edinburgh, UK, 2000, pp.347--355. \REF{[32]} Harrison R, Badoo N, Barry E \it et al. \rm Directions and methodologies for empirical software engineering research. \it Empirical Software Engineering, \rm 1999, 4(4): 405--410. \REF{[33]} Sjoberg D I K, Hannay J E, Hansen O \it et al. \rm A survey of controlled experiments in software engineering. \it IEEE Trans. Software Engineering, \rm 2005, 31(9): 733--753. \REF{[34]} Fenton N, Pfleeger S L, Glass R L. Science and substance: A challenge to software engineering. \it IEEE Software, \rm July 1994, pp.88--95. \REF{[35]} Zelkowitz M V, Wallance D R. Experimental models for validating technology. \it IEEE Software, \rm May 1998, pp.23--31. \REF{[36]} Rothermel G, Elbaum S, Malishevsky A G \it et al. \rm On test suite composition and cost-effective regression testing. \it ACM Trans. Software Engineering and Methodology, \rm 2004, 13(3): 277--331. \REF{[37]} Juristo N, Moreno A M, Vegas S. Reviewing 25 years of testing technique experiments. \it Empirical Software Engineering, \rm 2004, 9: 7--44. \REF{[38]} Zendler A. A preliminary software engineering theory as investigated by published experiments. \it Empirical Software Engineering, \rm 2001, 6: 161--180. \REF{[39]} IEEE. IEEE Standard Glossary of Software Engineering Terminology. IEEE Std 610.12-1990, 1990. \REF{[40]} Kitchenham B A, Pfleeger S L, Pickard L M \it et al. \rm Preliminary guidelines for empirical research in software engineering. \it IEEE Trans. Software Engineering, \rm 2002, 28(8): 721--734. \REF{[41]} Cater-Steel A, Toleman M, Rout T. Addressing the challenges of surveys in software engineering. In \it Proc. 2005 Int. Symp. Empirical Software Engineering, \rm 2005, pp.204--213. \REF{[42]} Jedlitschka A, Pfahl D. Reporting guidelines for controlled experiments in software engineering. In \it Proc. 2005 Int. Symp. Empirical Software Engineering, \rm Noosa Heads, Australia, 2005, pp.95--104. \REF{[43]} Do H, Elbaum S, Rothermel G. Infrastructure support for controlled experimentation with software testing and regression testing techniques. In \it Proc. 2004 Int. Symp. Empirical Software Engineering, \rm Redondo Beach, CA, USA, 2004, pp.60--70. \REF{[44]} Boehm B \it et al. \rm Using empirical testbeds to accelerate technology maturity and transition: The SCRover experience. In \it Proc. 2004 Int. Symp. Empirical Software Engineering, \rm Redondo Beach, CA, USA, 2004, pp.117--126. \REF{[45]} Cai K Y, Chen L. Analyzing software science data with partial repeatability. \it Journal of Systems and Software, \rm 2002, 63: 173--186. \REF{[46]} Xie M. Software Reliability Modeling. World Scientific, 1991. \REF{[47]} Cai K Y. Software Defect and Operational Profile Modeling. Kluwer Academic Publishers, 1998. \REF{[48]} Pham H. Software reliability and cost models: Perspectives, comparison, and practice. \it European Journal of Operational Research, \rm 2003, 149: 475--489. \REF{[49]} Vokolos F I, Frankl R H. Empirical evaluation of the textual differencing regression testing technique. In \it Proc. the Int. Conf. Software Maintenance, \rm Bethesda, Maryland, USA, November 1998, pp.44--53. \REF{[50]} Rothermel G, Untch R H, Chu C \it et al. \rm Prioritizing test cases for regression testing. \it IEEE Trans. Software Engineering, \rm 2001, 27(10): 929--948. \REF{[51]} Grassi V, Mirandola R. Devivation of Markov models for effectiveness analysis of adaptable software architectures for mobile computing. \it IEEE Trans. Mobile Computing, \rm 2003, 4(2): 114--131. \REF{[52]} Goseva-Popstojanova K, Trivedi K. Architecture-based approach to reliability assessment of software systems. \it Performance Evaluation, \rm 2001, 45: 179--204. \REF{[53]} Cai K Y, Lei X F, Shi Y \it et al. \rm Partial-repeatability as a new form of uncertainty. Fuzzy Logic, Software Computing and Computational Intelligence, Yingming Liu, Guoqing Chen, Mingsheng Ying (eds.), \it Proc. 11th Int. Fuzzy Systems Association World Congress, \rm Tsinghua University Press/Springer, Beijing, 2005, pp.157--166. \REF{[54]} Whittaker J A. What is software testing? And why is it so hard? \it IEEE Software, \rm January/February, 2000, 17(1): 70--79. \REF{[55]} Cai K Y. Optimal software testing and adaptive software testing in the context of software cybernetics. \it Information and Software Technology, \rm 2002, 44: 841--855. \REF{[56]} Cai K Y, Li Y C, Liu K. Optimal and adaptive testing for software reliability assessment. \it Information and Software Technology, \rm 2004, 46: 989--1000. \REF{[57]} Guo Y L, Shen F J. The History of Physics. Tsinghua University Press, 1993. (in Chinese) \REF{[58]} Johannes Kepler. His life, his laws and times. http://kepler.nasa.gov/johannes/. \REF{[59]} Li M, Vitanyi P M B. An Introduction to Kolmogorov Complexity and Its Applications, Second Edition. Springer-Verlag, 1997. \REF{[60]} Wiener N. Cybernetics: Or Control and Communication in the Animal and the Machine. John Wiley \& Sons, 1948. \REF{[61]} Lyu M R (ed.). Handbook of Software Reliability Engineering. McGraw-Hill, 1996. \REF{[62]} Fenton N, Littlewood B, Neil M. Applying Bayesian belief network in systems dependability assessment. In \it Proc. Safety Critical Systems Symposium, \rm Springer-Verlag, Leeds, UK, 1996, pp.71--94. \REF{[63]} Valiant L G. A theory of the learnable. \it Communications of the ACM, \rm 1984, 27(11): 1134--1142. \REF{[64]} Angluim D. Queries revisited. \it Theoretical Computer Science, \rm 2004, 313: 175--194. \REF{[65]} C de la Higuera. A bibliographical study of grammatical inference. \it Pattern Recognition, \rm 2005, 38: 1332--1348. \REF{[66]} Cai K Y, Cangussu J W, DeCarlo R A \it et al. \rm An overview of software cybernetics. In \it Proc. the 11th Int. Workshop on Software Technology and Engineering Practice, \rm IEEE Computer Society Press, Amsterdam, Holland, 2003, pp.77--86. \REF{[67]} Belli F, Cai K Y, DeCarlo R A \it et al. \rm Introduction to the special section on software cybernetics. \it Journal of Systems and Software, \rm article in press, available online, 2006.