cs.AINov 17, 2026

Deep Reinforcement Learning for Heat Pump and Underfloor Heating Control: An End-to-End Case Study

Authors: Antonio Manjavacas, David Suárez-Caro, Alejandro Campoy-Nieves, Guillermo Rey, Gonzalo Satrústegui, Miguel Molina-Solana

Organizations: Department of Computer Science and Artificial Intelligence, Universidad de Granada, Granada, 18071, Spain · Georg Fischer Building Flow Solutions, Avda. Leonardo da Vinci 15-17-19, Getafe, 28906, Madrid, Spain

Abstract

Efficient control of underfloor heating (UFH) systems coupled with heat pumps (HPs) is challenging due to the high thermal inertia of buildings and the slow dynamic response of UFH systems. Deep Reinforcement Learning (DRL) offers a data-driven control approach capable of simultaneously optimizing energy consumption and thermal comfort. However, the application of DRL-based controllers in occupied buildings under real operating conditions remains limited. This work presents and end-to-end case study (development, evaluation, and real-world deployment) of a DRL-based controller for an integrated UFH-HP system. First, a simulation-based pilot study was conducted to assess different DRL algorithms, control strategies, and configurations. Subsequently, a digital twin of a real residential building was developed and used to train DRL agents under simulated operating conditions. The best-performing agent was then deployed in the physical building without additional online fine-tuning. The results show that the controller successfully maintained indoor temperatures within the prescribed comfort limits while dynamically adjusting the HP supply temperature and UFH valve positions. These findings demonstrate the feasibility of deploying DRL controllers in occupied buildings and highlight their robustness and the practical challenges associated with transferring control policies from digital environments to real heating systems.

Figures & tables

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. When does deep reinforcement learning pay off in HVAC supervisory control? Four small Modelica case studies trained directly on FMUs

    Nov 17, 2026Bingnan Zhao, Harshala Gammulle, Tharindu FernandoHeating, Ventilation, and Air Conditioning ControlBuilding Energy Management

  2. A Unified Python Framework for Direct PPO-based Control of AHUs with Economizer Logic and CO2-Constrained Ventilation

    May 23, 2026Erfan Haghighat Damavandi, Davide Papurello, Mahdi Alibeigi +3RL ControlConstrained RL

  3. Safe Deep Reinforcement Learning for Building Heating Control and Demand-side Flexibility

    Apr 17, 2026Colin Jüni, Mina Montazeri, Yi Guo +3RL ControlBuilding Energy Management