Exact and approximate Nash equilibria in discounted Markov stopping games with terminal redemption

This work is concerned with a class of discrete-time, zero-sum games with two players and Markov transitions on a denumerable space. The general structure of the game is as follows: Before the game starts player I receives a bond from player II and, at each decision time, player II can stop the system paying a terminal bond to player I or, if the game is not halted, player I selects an action to drive the system and receives a running reward bond from player II. At termination, player I redeems the accumulated bonds and the performance of a pair of decision strategies is measured by the expectation of the value at time zero of the bonds accumulated by player I. Under standard continuity-compactness conditions, the following conclusions are established: (i) it is shown that this stopping game has a value function which is characterized by an equilibrium equation, and (ii) such a result is used to obtain the existence of a Nash equilibrium. Also, (iii) the method of successive approximations is used to construct approximate Nash equilibria for the game.