Too Hard, Too Easy, Just Right: Performance-Driven Challenge Adaptation in Virtual Reality Firefighter Training

Xuan Wang; Jianan Jiang; Rongqian Chen; Tian Lan; Bin Li

cs.AIOct 28, 2026

Too Hard, Too Easy, Just Right: Performance-Driven Challenge Adaptation in Virtual Reality Firefighter Training

Authors: Xuan Wang, Jianan Jiang, Rongqian Chen, Tian Lan, Bin Li

Organizations: The Pennsylvania State University State College, PA, USA · The George Washington University Washington, DC, USA

Abstract

Effective firefighter training requires practice under dynamic and stressful emergency conditions, yet real-world live-fire exercises are costly, risky, and difficult to scale. Virtual Reality (VR) offers a safe and controllable alternative, but many existing VR firefighter training systems rely on fixed scenarios or provide limited difficulty adaptation, making it difficult to maintain an appropriate level of challenge for skill development as trainees' performance changes over time. To address this limitation, we present ProACT, a performance-driven adaptive training system that regulates scenario difficulty through closed-loop interactions between trainee performance and simulated fire dynamics. Specifically, ProACT combines a global module that continuously adjusts scene-level fire spread dynamics in real time based on trainee performance with a local module that periodically updates object-level flammability across the environment. Our evaluation shows that ProACT improves both objective training performance and subjective user experience compared with existing adaptive and non-adaptive baselines. We further discuss how adaptive immersive environments can extend beyond firefighting to broader training domains.

Figures & tables

Figure 1. (i) VR training workflow consists of four stages: ① Start , where trainees initiate the session; ② Environment , where trainees explore the scene and identify potential hazards; ③ Extinguishing , during which users actively suppress the spreading fire; and ④ Complete , which indicates successful task completion. (ii) System architecture illustrating how the trainee (human-in-the-loop) interacts with the immersive environment, while the AI-agent performs scene-level real-time regulation of fire spread dynamics and object-level periodic updates of flammability to maintain an appropriate challenge level during training. Four stages of the VR training workflow and the human-in-the-loop architecture linking the trainee, the virtual environment, and the global and local adaptation modules.

Figure 2. System overview of ProACT. The system monitors trainee behavior, environment states, and interaction performance during VR firefighter training, and adapts scenario difficulty through a multi-scale adaptive engine. The global module regulates scene-level fire spread probability in real time, while the local module periodically updates object-level flammability based on scene semantics and trainee behavior. These adjustments are fed back to the virtual environment to support continuous personalized training. Trainee behavior, environment states, and task performance feed a global fire-spread controller and a local semantic reasoning module, whose updates return to the VR environment.

Figure 3. System flow of the Global Fire Dynamics Regulation Module. The controller continuously measures the experience gap between the trainee’s actual progress and the target progression, and dynamically tunes the scene-level fire spread probability to avoid both overload and under-challenge. A feedback loop compares observed fire suppression with target progression and adjusts the global fire-spread probability.

Figure 4. Overview of the Local Semantic Scene Reasoning Module. The module combines static scene semantics, including object name, location, size, and material category, with dynamic trainee context, including current position and recent movement trajectory. An LLM-based reasoning agent uses these inputs to guide localized updates to the object-level flammability map for local environment regulation. Static object metadata and recent trainee context feed an LLM reasoning agent that selects regions for bounded flammability updates.

Figure 5. Simulation environments used for the staged evaluation. (a) The 2D grid-world abstracts fire propagation and trainee suppression on a homogeneous grid, isolating the effect of global challenge regulation; current fire cells at step $k$ , candidate spreading cells at $k+1$ , neighboring fire cells, and the simulated trainee are visualized. (b) The 3D object-aware simulation introduces scene geometry, object-level structure, and heterogeneous flammability, providing a more realistic setting for evaluating the coordination of global fire regulation and local semantic adaptation. Two simulation settings: a homogeneous 2D fire grid with a simulated trainee and a 3D scene containing objects with heterogeneous flammability.

Method	$\epsilon=0.05$			$\epsilon=0.20$			$\epsilon=0.40$			$\epsilon=0.60$
Method	OT	Mean	Std	OT	Mean	Std	OT	Mean	Std	OT	Mean	Std
StaticEasy	2.0	34.83	43.50	2.0	73.91	62.78	1.0	110.91	58.03	0.0	143.36	30.47
StaticHard	0.0	135.90	42.97	0.0	140.29	36.45	0.0	149.63	13.63	0.0	151.00	0.00
Reactive	54.0	63.22	16.87	74.0	72.67	16.43	75.0	95.63	21.81	31.0	124.75	28.62
ProACT	77.0	73.17	15.07	90.0	83.29	19.47	78.0	95.40	25.18	60.0	108.46	28.86

Table 1. On-target completion rate and completion-time statistics in the 2D grid-world simulation. OT denotes the on-target completion rate (% of episodes completed within the target window); Mean and Std denote the mean and standard deviation of completion time across 100 independent episodes. Bold : highest OT for each capability level; underline : second-highest OT.

Figure 6. Completion-time distributions in the 2D grid-world simulation across the four capability levels (

\epsilon=0.05,0.20,0.40,0.60

; larger

\epsilon

= less consistent behavior). Each panel aggregates 100 episodes; the shaded band marks the target window

[\underline{\tau},\overline{\tau}]

and the dashed line the target time

\tau^{*}

. ProACT achieves the highest on-target completion rate across all evaluated capability levels and concentrates more completions within the target window than the baselines. Completion-time distributions for four methods at four simulated capability levels in the 2D grid-world, with the target completion window shaded.

Figure 7. Representative fire-load trajectories in the 2D grid-world simulation. Each panel shows a matched episode across StaticEasy, StaticHard, Reactive, and ProACT under one simulated trainee capability level, with active fire load

A[k]

plotted over time using a linear y-axis. The shaded region denotes the target challenge window

[\underline{\tau},\overline{\tau}]

, and the vertical dashed line marks the target time

\tau^{*}

. Markers denote episode outcomes: circles indicate on-target completion, while crosses indicate off-target completion or termination at the 150-step horizon. Matched 2D fire-load trajectories for four methods and four capability levels, with markers indicating completion outcomes relative to the target window.

Method	$\epsilon=0.05$			$\epsilon=0.20$			$\epsilon=0.40$			$\epsilon=0.60$
Method	OT	Mean	Std	OT	Mean	Std	OT	Mean	Std	OT	Mean	Std
StaticEasy	16.0	71.77	42.04	40.0	80.14	40.43	65.0	98.52	34.09	23.0	134.53	23.72
StaticHard	41.0	103.95	42.26	35.0	119.38	37.30	12.0	141.13	22.39	1.0	150.31	6.87
Reactive	22.0	76.08	43.23	40.0	84.14	41.12	66.0	98.92	33.67	23.0	133.97	24.47
ProACT	49.0	108.34	37.41	48.0	107.58	37.78	61.0	93.66	35.66	70.0	98.10	30.88

Table 2. On-target completion rate and completion-time statistics in the 3D object-aware simulation. OT denotes the on-target completion rate (% of episodes completed within the target window); Mean and Std denote the mean and standard deviation of completion time across 100 independent episodes. Bold : highest OT for each capability level; underline : second-highest OT.

Figure 8. Completion-time distributions in the 3D object-aware simulation across the four capability levels (

\epsilon=0.05,0.20,0.40,0.60

; larger

\epsilon

= less consistent behavior). Each panel aggregates 100 episodes; the shaded band marks the target window and the dashed line the target time. ProACT keeps completions inside the window more consistently than StaticEasy, StaticHard, and Reactive. Completion-time distributions for four methods at four simulated capability levels in the 3D object-aware environment, with the target window shaded.

Figure 9. Representative fire-load trajectories in the 3D object-aware simulation. Each panel shows a matched episode across StaticEasy, StaticHard, Reactive, and ProACT under one simulated trainee capability level, with active fire load

A[k]

plotted over time using a symmetric logarithmic y-axis. The shaded region denotes the target challenge window

[\underline{\tau},\overline{\tau}]

, and the vertical dashed line marks the target time

\tau^{*}

. Markers denote episode outcomes: circles indicate on-target completion, while crosses indicate off-target completion or termination at the 150-step horizon. Matched 3D fire-load trajectories for four methods and four capability levels on a symmetric logarithmic scale, with completion outcomes marked.

Item	Construct	Statement
Q1	Engagement	I found the training scenario engaging and interesting.
Q2	Realism	The virtual fire environment felt realistic enough for me to respond as if the situation mattered.
Q3	Adaptation	The system appeared to adjust the fire behavior in response to my actions.
Q4	Trust	I trusted that the system adapted the difficulty in a reasonable way based on my performance.
Q5	Challenge	The difficulty level in this condition was appropriate for effective learning.
Q6	Effort	The mental effort required during this condition was appropriate—neither too high nor too low.

Table 3. Post-condition questionnaire items used in the user study.

Figure 10. Post-condition questionnaire ratings across the four conditions (Q1–Q8: engagement, realism, adaptation, trust, challenge, effort, relevance, reuse). Bars show means, error bars standard deviations, and dots individual participant ratings (7-point Likert scale). Ratings on eight questionnaire items for four study conditions, showing means, standard deviations, and individual participant responses.

Figure 11. Active fire-load progression over time in the user-study trials, one panel per condition. Thin gray lines are individual trials, colored lines the mean active fire count, and shaded bands $\pm 1$ standard deviation. The vertical shaded region marks the $60$ – $120$ s target window. ProACT generally drives the active fire load to zero within the target completion window; StaticEasy ends too early and StaticHard grows out of control. Active fire-load trajectories for the four user-study conditions, showing individual trials, mean curves, standard deviations, and the target completion window.

Adaptive Learning

Adaptive learning develops methods that adjust models, strategies, or instruction as data, environments, and learner needs change.

See all 107 papers

Fire as a Service: Augmenting Robot Simulators with Thermally and Visually Accurate Fire Dynamics
Mar 19, 2026Anton R. Wagner, Madhan Balaji Rao, Helge Wrede +2Robotics Simulation Synthetic Data Generation for Robotics
Department of Computer Science, Kiel University, Germany · Department of Computer Science, George Mason University, USA
PyroAdapt: Adapting Wildfire Prediction under Spatial Heterogeneity and Temporal Shift
May 12, 2026Enyi Jiang, Wu SunLearning to Rank Domain Adaptation
Siebel School of Computing and Data Science, University of Illiois at Urbana-Champaign · Computer Science, Stanford University · Department of Global Ecology, Carnegie Institution for Science
Evaluating the Impact of Adaptive Extended Reality on Human-Robot Interaction Across the Reality-Virtuality Continuum
Sep 25, 2026Carl Tornberg, Alicia Torck, Lotfi El Hafi +1Human-Robot Interaction Virtual Reality
Ritsumeikan University, 1-1-1 Noji-Higashi, Kusatsu, Shiga 525-8577, Japan · Université catholique de Louvain (UCLouvain), 1 Place de l’Université, Louvain-la-Neuve 1348, Belgium · Kyoto University, Yoshida-Honmachi, Sakyo, Kyoto 606-8501, Japan

Mar 19, 2026J/K move · Enter open · S save

Fire as a Service: Augmenting Robot Simulators with Thermally and Visually Accurate Fire Dynamics

Anton R. Wagner, Madhan Balaji Rao, Helge Wrede, Sören Pirk+1

Department of Computer Science, Kiel University, Germany · Department of Computer Science, George Mason University, USA

Robotics Simulation Synthetic Data Generation for Robotics Robotics Physics-Based Simulation

Open paper AskPDF arXiv

Most existing robot simulators prioritize rigid-body dynamics and photorealistic rendering, but largely neglect the thermally and optically complex phenomena that characterize real-world fire environments. For robots envisioned as future firefighters, this limitation hinders both reliable capability evaluation and the generation of representative training data prior to deployment in hazardous scenarios. To address these challenges, we introduce Fire as a Service (FaaS), a novel, asynchronous co-simulation framework that augments existing robot simulators with high-fidelity and computationally efficient fire simulations. Our pipeline enables robots to experience accurate, multi-species thermodynamic heat transfer and visually consistent volumetric smoke without disrupting high-frequency rigid-body control loops. We demonstrate that our framework can be integrated with diverse robot simulators to generate physically accurate fire behavior, benchmark thermal hazards encountered by robotic platforms, and collect realistic multimodal perceptual data. Crucially, its real-time performance supports human-in-the-loop teleoperation, enabling the successful training of reactive, multimodal policies via Behavioral Cloning. By adding fire dynamics to robot simulations, FaaS provides a scalable pathway toward safer, more reliable deployment of robots in fire scenarios.

Abstract

Adaptive Learning

Explore similar work

Fire as a Service: Augmenting Robot Simulators with Thermally and Visually Accurate Fire Dynamics

PyroAdapt: Adapting Wildfire Prediction under Spatial Heterogeneity and Temporal Shift

Evaluating the Impact of Adaptive Extended Reality on Human-Robot Interaction Across the Reality-Virtuality Continuum