Sample Path Sharing in Policy Improvement for Indoor Air Temperature Control

Stochastic simulation is a commonly used tool in building energy management and indoor air temperature control. We consider an HVAC system, where the speed of the fan is controlled to minimize a weighted sum of the energy cost and the thermal comfort of the occupants. The problem is formulated as a Markov decision process. Simulation-based policy improvement (SBPI) is used to improve from a given base policy. An important issue is to allocate the limited computing budget among action candidates to find the best action with the highest probability. We make two major contributions in this paper. First, we use event-based policies to handle the large state space. Second, we develop a sample path sharing method and combine it with equal allocation, successive rejects and optimal computing budget allocation. Numerical results show that, through sample path sharing, these methods can output policies with better performance than using them alone.