Supplier's optimal bidding strategy in electricity pay-as-bid auction: Comparison of the Q-learning and a model-based approach

In this paper, the bidding decision making problem in electricity pay-as-bid auction is studied from a supplier's point of view. The bidding problem is a complicated task, because of suppliers’ uncertain behaviors and demand fluctuation. In a specific case, in which, the market clearing price (MCP) is considered as a continuous random variable with a known probability distribution function (PDF), an analytic solution is proposed. The suggested solution is generalized to consider the effect of supplier market power due to transmission congestion. As a result, an algebraic equation is developed to compute optimal offering price. The basic assumption in this approach is to take the known probabilistic model for the MCP.The above-mentioned method, called model-based approach, is not more applicable in a realistic situation. In order to overcome the drawback of this method, which needs information about the MCP and its PDF, the supplier learns from past experiences using the Q-learning algorithm to find out the optimal bid price. The simulation results of the model-based and Q-learning methods are compared on a studied system. It is shown that a supplier using the Q-learning algorithm is able to find the optimal bidding strategy similar to one obtained by the model-based approach. Furthermore, to analyze a more realistic situation, the suppliers’ behaviors are modeled using a multi-agent system. Simulation results illustrate that the studied supplier finds the optimal bidding strategy in power market using the Q-learning algorithm.