Stationary equilibria in discounted stochastic games with weakly interacting players

We give sufficient conditions for a non-zero sum discounted stochastic game with compact and convex action spaces and with norm-continuous transition probabilities, but with possibly unbounded state space, to have a Nash equilibrium in homogeneous Markov strategies that depends in a Lipschitz continuous manner on the current state. If the underlying state space is compact this yields the existence of a stationary equilibrium. Stochastic games with weakly interacting players provide a probabilistic framework within which to study strategic behavior in models of non-market interactions.