Adaptive Generalized Policy Iteration in Active Fault Detection and Control★

The paper deals with a suboptimal solution to the problem of active fault detection and control for stochastic nonlinear systems over an infinite time horizon. The design of an active fault detector and controller is formulated as a dynamic optimization problem that is solved using the generalized policy iteration algorithm. A key parameter of this algorithm that influences computational demands and the speed of convergence is the number of successive approximations used in a policy evaluation step. Although general guidelines are known, no exact algorithm for choosing this parameter exists. The paper proposes an adaptive algorithm for determining this key parameter to speed up convergence given a specified accuracy of the solution. The adaptive algorithm is demonstrated and compared with non-adaptive generalized policy iteration algorithms in a numerical example.