Bargaining with non-convexities

We consider the canonical non-cooperative multilateral bargaining game with a set of feasible payoffs that is closed and comprehensive from below, contains the disagreement point in its interior, and is such that the individually rational payoffs are bounded. We show that a pure stationary subgame perfect equilibrium having the no-delay property exists, even when the space of feasible payoffs is not convex. We also have the converse result that randomization will not be used in this environment in the sense that all stationary subgame perfect equilibria do not involve randomization on the equilibrium path. Nevertheless, mixed strategy profiles can lead to Pareto superior payoffs in the non-convex case.