Software field failure rate prediction before software deployment

For both in-house development and outsourcing development environments, knowing the field failure rate of an integrated software system prior to field deployment provides guidance for better decision-makings in balancing reliability, time-to-market and development cost. This paper demonstrates a field failure rate prediction methodology that starts with analyzing system test data and field data (of previous releases or products) using software reliability growth models (SRGMs). A typical issue associated with predicting field failure rate based on test data is that potentially the test environment might not match exactly up the field environment. We discuss how to address the mismatch of the operational profiles of the test and filed environments. Two other practical issues in predicting field failure rates include that fault removals in the field are usually non-instantaneous and fixes of certain faults reported in the field can be deferred. Non-instantaneous fault removal and fault fix deferral becomes more realistic as the current software development environment shifts to a new trend of leveraging third-party, off-the-shelf, and semi-custom hardware and software and having the suppliers focus on development of highest-value applications and system integration. In such an environment, removing a fault might require a longer time and fix deferrals of certain faults becomes more possible in particular for the faults whose fixes will result in changes to other software components. In this paper, we illustrate how to incorporate these issues into field failure rate prediction. Confidence intervals of the predicted failure rate are also included to account for variations in the parameter estimation. Sensitivity analyses are conducted to estimate the uncertainties in the field failure rate prediction.