Optimal stopping of two-time scale Markovian systems: Analysis, numerical methods, and applications

Numerous systems arising in applications are subject to uncertainty and stochastic influence. They are often of large scales and have complex structures. They may also display hybrid behavior represented by regime-switching dynamic systems. Exact optimal control policies for these systems are difficult or virtually impossible to obtain. In this paper, we study a representative of such systems, which is an optimal stopping problem driven by a Markov chain. We consider the case that the chain has a large state space. Then, it is natural to divide the states into a number of groups so that the chain jumps frequently within each group and only occasionally among different groups. We develop a two-time-scale approach to reduce the overall dimensionality and construct near-optimal strategies for the original problem. Examples are provided to illustrate the results. Application examples to equity markets are also considered as well.