A randomized point-based value iteration POMDP enhanced with a counting process technique for optimal management of multi-state multi-element systems

Optimal decision-making for systems in the presence of uncertainty poses a significant challenge in many fields of research and for many applications. While Markov Decision Process (MDP) is a capable probabilistic framework to incorporate uncertainties in system behavior, measurement randomness arising from imperfect inspections is disregarded in those models. Additionally, the decision-making problem for multi-state multi-element systems has exponential time complexity with respect to the number of system elements. This paper introduces a new decision-making framework for such systems that incorporates element-level decision variables and their consequences at the system-level of an asset. The framework employs a Partially Observable MDP (POMDP) with a randomized point-based value iteration solution strategy to capture system forecasting uncertainty as well as randomness in inspection measurements. The capability of the framework to handle large-scale optimization problems for element-level decision-making in multi-element systems is considerably enhanced via a counting process state reduction technique that is introduced and integrated into the POMDP model. The application of the proposed framework is demonstrated for long-run decision-making regarding maintenance, rehabilitation, and repair of a bridge system with realistic settings. Based on numerical results, it is concluded that the proposed framework composed of POMDP and the counting process techniques provides an efficient yet accurate approach for the optimal management of multi-state multi-element systems.