TUNEOPT: An Evolutionary Reinforcement Learning HVAC Controller For Energy-Comfort Optimization Tuning

HVAC systems account for the majority of energy consumption in buildings. Efficient control of HVAC systems can reduce energy consumption and enhance occupants’ comfort. In the existing literature, energy-comfort or cost-comfort co-optimization frameworks commonly involve manual tuning of the balancing coefficient between energy and comfort through parameter tuning by an expert. Nevertheless, achieving the optimal balance between energy usage and occupant comfort remains challenging. This limitation restricts the generalizability of different formulations across various scenarios or testing on different environments. In this paper, we propose an implicit evolutionary Reinforcement Learning (RL) approach to learn and adapt the trade-off parameter of an energy-comfort optimization formulation. We have developed a predictive comfortenergy co-optimization formulation for controlling the setpoint of a building. The RL agent utilizes a novel guidance-induced random search method to learn the energy-comfort trade-off coefficient and guide the optimization formulation. The reward function of the RL model is energy productivity (comfort over energy consumption). To evaluate the feasibility of our proposed approach, we conducted experiments on a real-world testbed - i.e., an apartment unit. Our feasibility study shows that the proposed approach can learn an optimal control parameter and reduce energy consumption by 24.3% while decreasing comfort by only 1% compared to the baseline.