A Multi-step Q(Λ) learning Approach to Power System Stabilizer

Power system stabilizers (PSS) are used to generate supplementary control signals for excitation system in order to damp the low frequency power system oscillation. To overcome the drawbacks of conventional PSS (CPSS), numerous techniques have been proposed in the literature. This paper proposes an optimal control methodology based on reinforcement learning (RL) for damping the low frequency power system oscillations. Q(Λ) learning is a relative new model-free RL algorithm that extends the one-step Q-learning by combining it with TD(Λ) returns for general Λ in a incremental way for the delayed RL problem. The Q(Λ) learning makes use of TD(Λ) returns as the value function estimator through a trace mechanism, and additional traces appear to help bridge the gap between frequency and recency information for heuristic events. In this paper, rotor speed deviation is adopted as the state feedback input, and the voltage deviation, power deviation and rotor speed deviation are formulated as reward function. The case study shows that both of them are very helpful to enhance the small-disturbance dynamics of power system. The multi-step Q(Λ) learning can enhance convergence rate of reinforcement learning algorithm.