Time vs. ensemble averages for nonstationary time series

We analyze whether sliding window time averages applied to stationary increment processes converge to a limit in probability. The question centers on averages, correlations, and densities constructed via time averages of the increment x(t,T)=x(t+T)−x(t), e.g. x(t,T)=ln(p(t+T)/p(t)) in finance and economics, where p(t) is a price, and the assumption is that the increment is distributed independently of t. We apply Tchebychev's Theorem to the construction of statistical ensembles, and then show that the convergence in probability condition is not satisfied when applied to time averages of functions of stationary increments. We further show that Tchebychev's Theorem provides the basis for constructing approximate ensemble averages and densities from a single, historic time series where, as in FX markets, the series shows a definite 'statistical periodicity'. The convergence condition is not satisfied strongly enough for densities and certain averages, but is well-satisfied by specific averages of direct interest. Rates of convergence cannot be established independently of specific models, however. Our analysis shows how to decide which empirical averages to avoid, and which ones to construct.