From Thermal Preference Prediction to Adaptive Thermal Intervention: A Reinforcement Learning Approach Using Physiological and Environmental Sensing

Isibor Kennedy Ihianle; Emmanuel Manu; Ehsan Asnaashari; Mojgan Jadidi; Pedro Machado; Amrit Sagoo; Ahmad Lotfi

cs.LGAug 19, 2026

From Thermal Preference Prediction to Adaptive Thermal Intervention: A Reinforcement Learning Approach Using Physiological and Environmental Sensing

Authors: Isibor Kennedy Ihianle, Emmanuel Manu, Ehsan Asnaashari, Mojgan Jadidi, Pedro Machado, Amrit Sagoo, Ahmad Lotfi

Organizations: Department of Computer Science, Nottingham Trent University, Nottingham, NG11 8NS, United Kingdom · York University, Ontario, M3J 1P3, Canada

Abstract

Personalised thermal comfort is essential for occupant wellbeing and for the development of more responsive building-control strategies, yet conventional Heating, Ventilation, and Air Conditioning (HVAC) systems rely on static setpoints and population-level comfort models that fail to capture individual physiological variability. This paper presents a two-stage personalised thermal comfort approach integrating multimodal physiological and environmental sensing with reinforcement learning-based decision-making.

Figures & tables

Figure 1: Proposed two-stage personalised thermal comfort architecture. Stage 1 develops a personalised comfort oracle for thermal preference prediction using physiological, body-proximity and weather-derived features. Stage 2 embeds the comfort oracle within a reinforcement learning control module to recommend an equivalent environmental-temperature intervention that maximises predicted comfort while penalising unnecessary or excessive control actions.

Panel of Figure 2 — Figure 2: Comparison of the candidate Stage 1 Comfort Oracle models across the four feature representations. Mean prediction performance is reported using (a) Accuracy, (b) Macro F 1 -score and (c) Cohen’s $\kappa$ , averaged over all participants. Phys denotes physiological features only; Phys+60 denotes physiological features with 60-minute environmental descriptors; Phys+480 denotes physiological features with 480-minute environmental descriptors; and Phys+60+480 combines both environmental contexts.

Model

0.61/ 0.39/ 0.20

0.61/ 0.32/ 0.04

0.65/ 0.39/ 0.15

0.84/ 0.46/ -0.09

0.84/ 0.42/ 0.27

0.59/ 0.35/ 0.15

0.75/ 0.34/ 0.11

0.89/ 0.31/ 0.00

0.82/ 0.41/ 0.23

0.69/ 0.34/ 0.07

0.88/ 0.31/ -0.03

0.53/ 0.51/ 0.08

0.68/ 0.55/ 0.14

0.83/ 0.30/ 0.00

0.68/ 0.43/ 0.32

0.59/ 0.54/ 0.18

0.69/ 0.49/ 0.29

0.91/ 0.32/ -0.01

0.82/ 0.42/ 0.23

0.39/ 0.27/ 0.06

0.70/ 0.37/ 0.11

0.89/ 0.40/ 0.17

0.76/ 0.35/ 0.05

0.67/ 0.43/ 0.16

0.86/ 0.42/ 0.10

0.62/ 0.61/ 0.25

0.66/ 0.59/ 0.18

0.83/ 0.36/ 0.11

0.65/ 0.43/ 0.30

0.53/ 0.41/ 0.07

0.65/ 0.42/ 0.18

0.78/ 0.44/ -0.11

0.85/ 0.47/ 0.38

0.53/ 0.38/ 0.21

0.70/ 0.34/ 0.06

0.89/ 0.31/ 0.00

0.78/ 0.35/ 0.06

0.67/ 0.39/ 0.11

0.86/ 0.31/ -0.04

0.65/ 0.64/ 0.31

0.68/ 0.52/ 0.10

0.83/ 0.30/ 0.00

Table 1: Participant-level Stage 1 performance for Physiology + Env480. Values are Accuracy/Macro F 1 /Cohen’s

\kappa

FeatureShort	Phys	Phys+60	Phys+480	Phys+60+480
ID
1	GB: 0.68/0.45/0.36	ET: 0.58/0.39/0.17	ET: 0.65/0.43/0.30	ET: 0.55/0.36/0.08
2	ET: 0.39/0.36/-0.05	ET: 0.49/0.37/-0.00	ET: 0.53/0.41/0.07	ET: 0.57/0.41/0.05
3	ET: 0.62/0.38/0.11	GB: 0.69/0.51/0.31	GB: 0.69/0.49/0.29	GB: 0.69/0.49/0.29
4	RF: 0.93/0.48/0.00	RF: 0.87/0.47/-0.07	RF: 0.84/0.46/-0.09	RF: 0.82/0.45/-0.10
5	GB: 0.85/0.47/0.38	GB: 0.82/0.42/0.23	GB: 0.82/0.42/0.23	GB: 0.80/0.42/0.20

Table 2: Selected personalised Comfort Oracle for each participant and feature representation. Values are Model: Accuracy/Macro F 1 /Cohen’s

\kappa

Comparison	Metric	Test	Statistic	$p$ -value
Feature sets	Accuracy	Friedman	0.250	0.969
Feature sets	Macro F 1	Friedman	4.887	0.180
Feature sets	Cohen’s $\kappa$	Friedman	2.758	0.430
Models	Accuracy	Friedman	1.815	0.404
Models	Macro F 1	Friedman	2.286	0.319
Models	Cohen’s $\kappa$	Friedman	3.857	0.145

Table 3: Statistical comparison of Stage 1 prediction performance using Friedman tests across participants.

Oracle	Controller	Best Variant	Comfort	Reward	HVAC $\Delta T$	Reward/Energy	Comfort/Energy	Decrease	No Change	Increase
Phys+60	CB	Bandit Comfort	0.700	89.942	3.342	26.917	0.210	0.093	0.817	0.090
Phys+60	QL	QL alpha=0.16	0.668	84.442	2.628	32.132	0.254	0.019	0.978	0.003
Phys+60	DQN	DQN lr=0.0002	0.678	84.382	3.343	25.238	0.203	0.417	0.386	0.198
Phys+480	CB	Bandit Comfort	0.684	87.417	3.602	24.269	0.190	0.084	0.781	0.134
Phys+480	QL	QL alpha=0.16	0.658	82.764	2.642	31.324	0.249	0.019	0.978	0.003
Phys+480	DQN	DQN lr=0.0001	0.664	82.374	3.225	25.542	0.206	0.477	0.427	0.096

Table 4: Best Stage 2 controller variants across oracle feature representations. The best variant within each controller family was selected using mean reward.

Panel of Figure 4 — Figure 4: Comparison of representative closed-loop proxy thermal-policy behaviour across two subjects under Phys+480. In the plotted legends, “Actual room temp” denotes the observed temperature proxy derived from the dataset, while “Recommended room temp” denotes the controller-generated equivalent environmental-temperature proxy. These trajectories do not represent measured room-temperature regulation or physical HVAC actuation.

Panel of Figure 5 — Figure 5: Comparison of representative closed-loop proxy thermal-policy behaviour across two subjects under Phys+60. In the plotted legends, “Actual room temp” denotes the observed temperature proxy derived from the dataset, while “Recommended room temp” denotes the controller-generated equivalent environmental-temperature proxy. These trajectories do not represent measured room-temperature regulation or physical HVAC actuation.

Panel of Figure 6 — Figure 6: Comparison of representative closed-loop proxy thermal-policy behaviour across two subjects under Phys+60+480. In the plotted legends, “Actual room temp” denotes the observed temperature proxy derived from the dataset, while “Recommended room temp” denotes the controller-generated equivalent environmental-temperature proxy. These trajectories do not represent measured room-temperature regulation or physical HVAC actuation.

Oracle	Controller	Best Variant	Accuracy	Balanced Accuracy	Macro F1	Cohen’s kappa
Phys+60	CB	Bandit Comfort	0.622	0.369	0.367	0.043
Phys+60	QL	QL alpha=0.16	0.691	0.332	0.285	-0.012
Phys+60	DQN	DQN lr=0.0002	0.442	0.458	0.395	0.129
Phys+480	CB	Bandit Comfort	0.604	0.368	0.374	0.045
Phys+480	QL	QL alpha=0.16	0.688	0.327	0.278	-0.021
Phys+480	DQN	DQN lr=0.0001	0.412	0.392	0.328	0.052

Table 5: Action agreement performance of the selected Stage 2 controllers. Only the best variant from each controller family, as selected in Table 4 , is reported.

Oracle	Metric	Test	N	Statistic	$p$ -value
Phys+60	Comfort	Friedman	14	18.43	0.0000996
Phys+60	Reward	Friedman	14	22.29	0.0000145
Phys+60	Reward/Energy	Friedman	14	15.86	0.0003603
Phys+60	HVAC $\Delta T$	Friedman	14	18.43	0.0000996
Phys+60	Action switch rate	Friedman	14	19.13	0.0000701
Phys+480	Comfort	Friedman	14	20.76	0.0000310

Table 6: Friedman statistical comparison of the selected Stage 2 controllers across participants for each oracle feature representation.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Table A.1 : Participant-level Stage 1 performance across all feature representations. Values are Accuracy/Macro F 1 /Cohen’s

\kappa

Oracle	Controller	Variant	Comfort	Reward	HVAC $\Delta T$	Reward/Energy	Comfort/Energy	Decrease	No Change	Increase
Phys+60	CB	Bandit Comfort	0.700	89.942	3.342	26.917	0.210	0.093	0.817	0.090
Phys+60	CB	Bandit Balanced	0.698	89.344	3.202	27.900	0.218	0.087	0.838	0.075
Phys+60	CB	Bandit Energy	0.695	88.703	3.100	28.616	0.224	0.081	0.855	0.064
Phys+60	QL	QL alpha=0.16	0.668	84.442	2.628	32.132	0.254	0.019	0.978	0.003
Phys+60	QL	QL alpha=0.08	0.667	84.314	2.712	31.087	0.246	0.031	0.960	0.009
Phys+60	QL	QL alpha=0.05	0.667	84.112	2.708	31.059	0.246	0.031	0.954	0.015

Appendix

Table A.2 : Stage 2 controller performance across all oracle feature representations and controller variants.

Heating, Ventilation, and Air Conditioning Control

Heating, ventilation, and air conditioning control optimizes building energy use while maintaining thermal comfort, indoor air quality, and equipment safety.

See all 21 papers

Generalizing HVAC Control With Domain Randomized Reinforcement Learning
Sep 5, 2026Pablo Boitel, Kun ZhangHeating, Ventilation, and Air Conditioning Control RL Control
École de technologie supérieure (ÉTS) Montréal, Québec, Canada
ADAPT: Physics-Aware Diffusion-based World Models for Adaptive Predictive Transferable HVAC Control
Aug 20, 2026Xu Yang, Kailai Sun, Dianyu Zhong +1Heating, Ventilation, and Air Conditioning Control Building Energy Management
Department of Automation Tsinghua University, Beijing, China · SMART, Singapore Massachusetts Institute of Technology Cambridge, MA, USA
ThermoLLM: Thermodynamics-Aware HVAC Control with Spatial-Semantic Knowledge Graph
Jun 22, 2026Kirtan Bhatt, Xiachong Lin, Matthew Amos +2Language Model-Based Control Building Energy Management
UNSW Sydney · Sydney, NSW, Australia · University of New South Wales +2

Sep 5, 2026J/K move · Enter open · S save

Generalizing HVAC Control With Domain Randomized Reinforcement Learning

Pablo Boitel, Kun Zhang

École de technologie supérieure (ÉTS) Montréal, Québec, Canada

Heating, Ventilation, and Air Conditioning Control RL Control Domain Randomization Robust RL Physics-Informed RL

Open paper AskPDF arXiv

Deploying advanced HVAC (Heating, Ventilation and Air Conditioning) controllers at scale remains difficult because performance often depends on accurate building models or per-site retuning. We propose NOMAD-RL (Neural Online Meta-Adaptation for Dynamics), a general-purpose Reinforcement Learning (RL) controller designed to transfer across heterogeneous thermal zones through a universal, non-invasive thermostat interface. The controller acts on temperature setpoints from zone measurements and forecasts, while a recurrent policy supports online adaptation under partial observability. Our main contribution is an adaptive domain randomization scheme based on physics-informed normalizing flows, which models correlated and multimodal distributions of thermal-zone parameters while maintaining physical plausibility and controllability. This produces a realistic and progressively adaptive training curriculum that improves transfer across buildings. We evaluate NOMAD-RL against a constant-setpoint PID controller, RL without domain randomization, and MPC in single- and multi-zone settings. NOMAD-RL consistently outperforms the PID and non-randomized RL baselines, and approaches the performance of a well-tuned MPC, especially in the more challenging multi-zone case. These results highlight the potential of adaptive, physics-informed domain randomization for robust and transferable HVAC control.

Abstract

Heating, Ventilation, and Air Conditioning Control

Explore similar work

Generalizing HVAC Control With Domain Randomized Reinforcement Learning

ADAPT: Physics-Aware Diffusion-based World Models for Adaptive Predictive Transferable HVAC Control

ThermoLLM: Thermodynamics-Aware HVAC Control with Spatial-Semantic Knowledge Graph