Numerical Methods for Optimal Controls for Nonlinear Stochastic Systems With Delays: Algorithms and Data*

The Markov chain approximation numerical methods, widely used to compute optimal value functions and controls for stochastic systems, was extended to general controlled nonlinear (and possibly reflected) diffusions with delays in a recent book of the author, and the convergence of many types of algorithms was proved. The path, control and/or reflection terms can all be delayed. If the control and/or reflection terms are delayed, the memory requirements can be huge. Recasting the problem in terms of a “wave equation” yields algorithms with considerably reduced memory needs. We concentrate on the algorithms and present data showing that the methods work well.