Markov decision processes with state-dependent discount factors and unbounded rewards/costs

This paper deals with discrete-time Markov decision processes with state-dependent discount factors and unbounded rewards/costs. Under general conditions, we develop an iteration algorithm for computing the optimal value function, and also prove the existence of optimal stationary policies. Furthermore, we illustrate our results with a cash-balance model.

► This paper studies Markov decision processes with state-dependent discount factors. ► Suitable conditions are given for the existence of optimal stationary policies. ► Then, an iteration algorithm for computing the optimal value function is developed. ► Finally, a cash balance model is used to illustrate the main results in this paper.