Some successful approaches to software reliability modeling in industry

Over the past three years, we have been actively engaged in both software reliability growth modeling and architecture-based software reliability modeling for projects at Lucent Technologies. Our goal has been to include software into the overall reliability evaluation of a product design using either or both of these two fundamentally different approaches. During the course of our application efforts to real projects, we have identified practical difficulties with each approach. The application of software reliability growth models, for example, is plagued by widespread use of ad hoc test environments, and the use of architecture-based software reliability models is plagued by a large number of unknown parameters. In this paper, we discuss our methods for overcoming these and other practical difficulties. In particular, we show how calibration factors can be defined and used to adjust for the mismatch between the test and operational profiles of the software. We also present two useful ways to do sensitivity analyses that help alleviate the problem of so many uncertainties in the architecture-based modeling approach. We illustrate our methods with case studies, and offer comments on further work that is required to more satisfactorily bridge the gap between theory and applications in this research area.