Extremum seeking of dynamical systems via gradient descent and stochastic approximation methods

This paper examines the use of gradient based methods for extremum seeking control of possibly infinite-dimensional dynamic nonlinear systems with general attractors within a periodic sampled-data framework. First, discrete-time gradient descent method is considered and semi-global practical asymptotic stability with respect to an ultimate bound is shown. Next, under the more complicated setting where the sampled measurements of the plant’s output are corrupted by an additive noise, three basic stochastic approximation methods are analysed; namely finite-difference, random directions, and simultaneous perturbation. Semi-global convergence to an optimum with probability one is established. A tuning parameter within the sampled-data framework is the period of the synchronised sampler and hold device, which is also the waiting time during which the system dynamics settle to within a controllable neighbourhood of the steady-state input–output behaviour.