Optimal Control for Deterministic Hybrid Systems using Dynamic Programming

Dynamic programming provides a method to solve hybrid optimal control problems. This contribution extends existing numerical methods originally developed for purely continuous systems, to a class of hybrid systems with autonomous as well as controlled switching behavior. The hybrid dynamics is approximated by a locally consistent discrete Markov decision process. The original optimal control problem is then reformulated for the Markov decision process and solved by standard dynamic programming methods. The convergence of the discrete approximation to the original problem is ensured. The viability of the numerical scheme is illustrated by a two gear transmission system used previously in literature.