Adaptive critic methods for stochastic systems with input-dependent noise

In this paper, a novel dual heuristic programming (DHP) adaptive-critic-based cautious controller is proposed. The proposed controller avoids the pre-identification training phase of the forward model by taking into consideration model uncertainty when calculating the control law. It is suitable for linear and nonlinear, deterministic and stochastic control systems that are characterized by functional uncertainty. Convergence of the proposed DHP adaptive critic method to the correct value of the cost function is proven by evaluating analytically the correct value of the cost function, which satisfies the Bellman equation, and compares it to that calculated by the proposed method in a simple linear quadratic example. Moreover, the performance of the proposed cautious controller is demonstrated on linear one-dimensional and multidimensional examples.