Stochastic approximation with ‘controlled Markov’ noise

Stochastic approximation algorithms with additional noise that can be modelled as a controlled Markov process are analyzed and shown to track the solutions of a differential inclusion defined in terms of the ergodic occupation measures associated with the controlled Markov process.