Optimal Scheduling of the Maintenance and Improvement for Water Main System Using Markov Decision Process

Optimal scheduling of maintenance and improvement on the water main system is a substantial problem since it can greatly reduces the potential threat of failure and the expenses. The Markov decision process (MDP) based methodology to find the optimal schedule which minimizes the cost is proposed. Since it needs the information about the current state of pipe, cost, and deterioration model, the definition and the usage of auxiliary information are also presented. The objective function and detailed algorithm of dynamic programming (DP) are modified due to the difficulty of implementing the conventional DP approaches. The result is compared to two simple policies via Monte Carlo simulation. Validity of the solution and improvement in computational time are proved. Proposed decision framework provides an easy way to obtain the comprehensive and local-specialized policy.