Global convergence of discrete-time inhomogeneous Markov processes from dynamical systems perspective

Given the continuous real-valued objective function f and the discrete time inhomogeneous Markov process Xt defined by the recursive equation of the form Xt+1=Tt(Xt,Yt), where Yt is an independent sequence, we target the problem of finding conditions under which the Xt converges towards the set of global minimums of f. Our methodology is based on the Lyapunov function technique and extends the previous results to cover the case in which the sequence f(Xt) is not assumed to be a supermartingale. We provide a general convergence theorem. An application example is presented: the general result is applied to the Simulated Annealing algorithm.