Almost sure convergence of extremum seeking algorithm using stochastic perturbation

This paper presents an extremum seeking (ES) algorithm where the perturbation signal is a martingale difference sequence (m.d.s.) with a vanishing variance. The measurement noise at the plant output is modeled by a superposition of deterministic component, and a non-stationary colored noise signal. The optimizing set point of the uncertain reference-to-output equilibrium map is estimated by a stochastic approximation (SA)-type algorithm. The algorithm has a vanishing gain sequence dependent on the set point estimates. By utilizing powerful tools of the martingale convergence theory it is proved that with probability one the set point estimates converge to the optimizing equilibrium point, in spite of the presence of a measurement noise. This result is derived without requiring boundedness or any prior condition on the set point estimates.