Optimal choice for finite and infinite horizons

This paper lays down conceptual groundwork for optimal choice in infinite-horizon finite-state Markov decision problems. We distinguish two notions of a strategy being favored on the limit of horizons, and examine the properties of the emerging binary relations. After delimiting two senses of optimality, we analyze the relationship between the resulting sets of optimal strategies-including the Ramsey-Weizsäcker overtaking criterion-and their existence properties. We also relate to the pointwise limits of strategies of Fudenberg and Levine [3].