Imprecise stochastic processes in discrete time: global models, imprecise Markov chains, and ergodic theorems

•We develop global models for imprecise stochastic processes in discrete time.•We define joint lower and upper expectations and study their properties.•We do not impose the usual restriction that random variables should be bounded.•We apply our results to study the special case of imprecise Markov chains.•We prove point-wise ergodic theorems for imprecise Markov chains.

We justify and discuss expressions for joint lower and upper expectations in imprecise probability trees, in terms of the sub- and supermartingales that can be associated with such trees. These imprecise probability trees can be seen as discrete-time stochastic processes with finite state sets and transition probabilities that are imprecise, in the sense that they are only known to belong to some convex closed set of probability measures. We derive various properties for their joint lower and upper expectations, and in particular a law of iterated expectations. We then focus on the special case of imprecise Markov chains, investigate their Markov and stationarity properties, and use these, by way of an example, to derive a system of non-linear equations for lower and upper expected transition and return times. Most importantly, we prove a game-theoretic version of the strong law of large numbers for submartingale differences in imprecise probability trees, and use this to derive point-wise ergodic theorems for imprecise Markov chains.