Researchers in Germany have proposed using a machine learning technique known as reinforcement learning to optimize the performance of heat pumps and reduce electricity consumption in apartment buildings. Their simulation showed that significant savings can be achieved without sacrificing comfort levels.
A German research group has proposed using reinforcement learning (RL) to optimize the use of heat pumps (HPs) for space heating in large apartment buildings.
RL is a form of machine learning that teaches agents to make optimal decisions in dynamic environments. RL agents learn from the environment by receiving rewards or punishments.
“A critical step in designing an RL approach for building control is the selection of a meaningful reward function,” the scientists explained. “This is mandatory to provide the agent with the necessary feedback to optimize his behavior. For space heating control, this reward function is often designed using the internal room temperature in combination with the occupants’ comfort bands.”
The research team created an environmental modeling of a residential heating network, including a hot water tank (HWT) as heat storage without the need for a building model, and modeled a 100 kW heat pump based on a temperature-dependent coefficient of performance (COP) curve. “The agent learns to operate under perfect foresight conditions and to rely on demand forecasts,” it further explains.
The RL-based business operation would not compromise on the comfort level and would meet the heat demand of residents at all times. It was trained via a Proximal Policy Optimization (PPO) algorithm, which is commonly used in the field of reinforcement learning by training a computer agent’s decision function to perform difficult tasks. “It applies a ‘proximal’ approach by introducing a clipping mechanism into the objective function,” the team pointed out.
Their analysis took into account parameters such as the total electricity cost, the electricity cost due to heat demand, the electricity cost due to heat loss, the total electricity consumption of the HP, the number of on/off status changes of the HP, the average electrical operating power of the HP, maximum electrical operating power of the HP, average SOC of the HWT and maximum SOC of the HWT.
They assumed that the experimental setup would be applied in an apartment complex with 100 residential units and 7,000 m2 of living space with a space heating requirement of 25-28 kWh/m2 per year.
The scientists said their assessment showed that the proposed setup can reduce electricity costs by around 35% as it takes advantage of the variable electricity price, while also reducing the total energy consumption of the heat pump by up to 15%.
“Moreover, we show that the intelligent operation of HPs does not use the full installed rated electrical power and therefore could reduce investment costs,” she added. “The robustness of the approach is demonstrated by performing ten independent training and testing cycles for all setups with reproducible results.”
The new technique was introduced in the article “Prediction-based and data-driven reinforcement learning for residential heat pump operation,” published in Applied energy. The research group consisted of scientists from the DLR Institute for Software Technology, the Carl von Ossietzky University Oldenburg and the Fraunhofer Institute for Production Technology and Advanced Materials (Fraunhofer IFAM).
“Improvements for further research could include increasing the complexity of the environment,” the academics said, referring to the future direction of their work.
This content is copyrighted and may not be reused. If you would like to collaborate with us and reuse some of our content, please contact: editors@pv-magazine.com.
