Coupled Control and Wireless World Models for Resilient Remote Robotic Control

H. P. Madushanka; Sumudu Samarakoon; Mehdi Bennis

cs.ROSep 4, 2026

Coupled Control and Wireless World Models for Resilient Remote Robotic Control

Authors: H. P. Madushanka, Sumudu Samarakoon, Mehdi Bennis

Organizations: Centre for Wireless Communications, University of Oulu, 90570 Oulu, Finland

Abstract

Remote robotic systems operating over wireless networks must maintain reliable control despite limited communication resources, changing channel conditions, and environmental disturbances.However, continuously transmitting high-dimensional sensory observations, such as camera images, increases communication overhead and energy consumption while reducing robustness under unreliable connectivity.To address these challenges, this paper proposes a resilient communication-aware remote robotic control framework based on coupled control and wireless Joint Embedding Predictive Architecture (JEPA) world models that jointly capture robot dynamics and wireless channel evolution from visual observations and a combination of raw and structured radio frequency (RF) representations based on spectrograms and Persistence Images(PIs).The learned latent representations enable predictive communication scheduling by jointly forecasting future robot states and wireless conditions, thereby reducing unnecessary uplink transmissions while maintaining reliable control performance.Furthermore, an adaptive resilience mechanism detects latent prediction discrepancies and efficiently adapts perception embeddings to accommodate wireless and visual environmental changes without retraining the complete control policy.The proposed framework is evaluated in a synchronized Gazebo-Robot Operating System (ROS)-Sionna robot-wireless simulation environment under diverse wireless propagation and perception perturbations.Experimental results demonstrate significant improvements in communication efficiency, robustness, and resilience while maintaining navigation performance compared with conventional Proportional Integral Derivative (PID), model-free Deep Q-Network (DQN), and predictive approaches based on Vision Transformers(ViTs).

Figures & tables

Fig. 1 : Proposed communication-aware remote robotic control framework based on coupled C-JEPA and W-JEPA .

Fig. 2 : Structured RF representation generation from the wireless observation sequence $\boldsymbol{G}_{t}$ . Three alternative input representations are considered for the W-JEPA : raw CSI $\boldsymbol{g}_{t}$ , spectrogram images $\boldsymbol{q}_{t}^{s}$ , and PIs $\boldsymbol{q}_{t}^{p}$ .

Fig. 3 : Perception degradation scenarios considered for adaptive resilience: (a) global perceptual degradation caused by illumination variations and (b) local perceptual degradation caused by partial visual occlusions.

Fig. 4 : Experimental setup of the proposed communication-aware robotic framework. (a) Overall communication-aware experimental environment with the robot, onboard RGB observations, and sensing uplink/control downlink communication. (b) Gazebo environment for robot simulation and visual perception. (c) Corresponding Sionna RT scene for physics-based wireless channel generation, synchronized through ROS .

Metric	Definition
$T_{c}=t_{f}^{\mathrm{sim}}-t_{0}^{\mathrm{sim}}$	Task (navigation) completion time
$N_{\mathrm{comm}}=\sum_{t=1}^{T}\rho_{t}$	Communication (uplink) rounds
$E_{\mathrm{tot}}=\sum_{i=1}^{N_{\mathrm{comm}}}P_{i}\Delta t$	Total transmission (uplink) energy
$R=\sum_{t=1}^{T}r_{t}$	Accumulated task reward

TABLE I : Evaluation metrics used to assess the frameworks.

Fig. 5 : Wireless evaluation scenarios employed to assess the robustness of the proposed communication-aware framework: (a) Additional scatterers are introduced into both the Gazebo and Sionna RT scenes to generate diverse multipath propagation conditions; and (b) The access-point handover is enforced to modify the geometry of the wireless environment.

Metric		Proposed	DQN	PID
Completion time (s)		$57.1\pm 1.0$	$58.6\pm 1.6$	$77.3\pm 0.3$
Total reward		$775.4\pm 4.4$	$770.8\pm 3.5$	$716.1\pm 2.0$
Latency (ms)	Uplink	$0.48\pm 0.02$	$0.86\pm 0.00$	$2.63\pm 0.05$
Latency (ms)	Downlink	$0.44\pm 0.00$	$0.44\pm 0.00$	$1.31\pm 0.02$
Total energy (J)		$4.74\pm 0.16$	$9.21\pm 0.32$	$11.19\pm 0.15$

TABLE II : The comparison of the overall performance under E1 .

Panel of Figure 8 — Fig. 6 : Task completion time under wireless channel perturbations: (a) carrier frequency shifted from 2.14 GHz to 1.9 GHz, (b) carrier frequency shifted from 2.14 GHz to 2.6 GHz, and (c) additional scatterers introduced into the propagation environment.

Panel of Figure 9 — Fig. 7 : Total uplink transmission energy under wireless channel perturbations: (a) carrier frequency shifted from 2.14 GHz to 1.9 GHz, (b) carrier frequency shifted from 2.14 GHz to 2.6 GHz, and (c) additional scatterers introduced into the propagation environment.

Fig. 8 : Prediction error comparison between raw CSI -based and structured representation-based W-JEPA models across multiple prediction horizons.

Panel of Figure 11 — Fig. 9 : Impact of temporal window length ( $\tau$ ) on uplink transmit-power prediction error for (a) spectrogram-based W-JEPA and (b) PI -based W-JEPA .

Spectrogram-based W-JEPA			PI-based W-JEPA
$L$	$\mathcal{L}_{P}$ (W)	$T_{\textbf{G}}$ (ms)	$m$	$\mathcal{L}_{P}$ (W)	$T_{\textbf{G}}$ (ms)
64	$0.022\pm 0.026$	$1.74$	5	$0.020\pm 0.021$	$98.84$
96	$0.014\pm 0.018$	$1.71$	10	$0.019\pm 0.020$	$99.9$
128	$0.022\pm 0.025$	$1.66$	15	$0.025\pm 0.033$	$102.03$
160	$0.024\pm 0.027$	$1.65$	20	$0.014\pm 0.013$	$104.46$
200	$0.024\pm 0.028$	$1.63$	25	$0.028\pm 0.030$	$103.45$

TABLE III : Impacts of representation lengths on power prediction errors (

\mathcal{L}_{P}

) and representation generation times (

T_{\textbf{G}}

Fig. 10 : Impact of transmitted token size on reward for the proposed framework and the ViT -based baseline.

Fig. 11 : Impact of consecutive uplink communication losses on the achieved reward for the proposed framework and the ViT -based baseline.

Fig. 12 : Impact of adaptive latent representation alignment on reward under global perceptual degradation caused by illumination changes.

Fig. 13 : Impact of historical data ratio (

r

) and adaptation epochs (

\epsilon

) on recovery from local visual obstruction.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Component	Specification
CPU	Intel 11th Gen Core i5-11600K @ 3.90 GHz
	(6 cores, 12 threads, up to 4.90 GHz)
GPU	NVIDIA GeForce RTX 2080 Ti
Memory	32 GB RAM
Storage	1 TB NVMe SSD
Operating System	Ubuntu 20.04 LTS

Appendix

TABLE IV : Hardware Configuration

Software Settings		Wireless Communication
Software	Version	Parameter	Value
ROS	Noetic	Carrier frequency	2.14 GHz
Gazebo	11	Bandwidth	20 MHz
Sionna RT	1.2.1	Subcarriers	16
Mitsuba	3	Base station	3 arrays of
Blender	4.3.2		$4\times 2$ antennas

Appendix

TABLE V : Software & Wireless Configurations

Control JEPA
Parameter	Value	Parameter	Value
Learning rate	$2\times 10^{-4}$	Discount factor ( $\gamma$ )	0.99
Batch size	32	KL loss scale ( $\beta$ )	0.5
Gradient clipping	100	KL balancing ( $\mu$ )	0.8
Latent ( $z_{t}$ ) dim.	32	Replay memory size	$10^{6}$
Wireless JEPA		RL

Appendix

TABLE VI : Hyperparameters

Robotic Control

Robotic control studies how machines use sensing and computation to produce reliable movement and interaction in changing environments.

See all 587 papers

ResWM: Residual-Action World Model for Visual RL
Mar 11, 2026Jseen Zhang, Gabriel Adineera, Jinzhou Tan +1Model-Based RL World Model Learning
University of California, San Diego USA · Texas A&M University-Commerce USA
World Dreamer v5-lite: A Multi-Robot World Action Model with One Dreamer
Oct 27, 2026Guoning Wu, Zijian Cai, Yuhang Zhang +2World Models for Robotics Robotics
World Agents
Preserving Unstable Modes Through Inverse Dynamics in JEPA World Models
Oct 6, 2026Leonardo F. Toso, Yann LeCun, James Anderson +1Representation Learning World Model Learning
Columbia University · New York University, AMI Labs · New York University

Mar 11, 2026J/K move · Enter open · S save

ResWM: Residual-Action World Model for Visual RL

Jseen Zhang, Gabriel Adineera, Jinzhou Tan, Jinoh Kim

University of California, San Diego USA · Texas A&M University-Commerce USA

Model-Based RL World Model Learning Visual RL

Open paper AskPDF arXiv

Learning predictive world models from raw visual observations is a central challenge in reinforcement learning (RL), especially for robotics and continuous control. Conventional model-based RL frameworks directly condition future predictions on absolute actions, which makes optimization unstable: the optimal action distributions are task-dependent, unknown a priori, and often lead to oscillatory or inefficient control. To address this, we introduce the Residual-Action World Model (ResWM), a new framework that reformulates the control variable from absolute actions to residual actions -- incremental adjustments relative to the previous step. This design aligns with the inherent smoothness of real-world control, reduces the effective search space, and stabilizes long-horizon planning. To further strengthen the representation, we propose an Observation Difference Encoder that explicitly models the changes between adjacent frames, yielding compact latent dynamics that are naturally coupled with residual actions. ResWM is integrated into a Dreamer-style latent dynamics model with minimal modifications and no extra hyperparameters. Both imagination rollouts and policy optimization are conducted in the residual-action space, enabling smoother exploration, lower control variance, and more reliable planning. Empirical results on the DeepMind Control Suite demonstrate that ResWM achieves consistent improvements in sample efficiency, asymptotic returns, and control smoothness, significantly surpassing strong baselines such as Dreamer and TD-MPC. Beyond performance, ResWM produces more stable and energy-efficient action trajectories, a property critical for robotic systems deployed in real-world environments. These findings suggest that residual action modeling provides a simple yet powerful principle for bridging algorithmic advances in RL with the practical requirements of robotics.

Abstract

Robotic Control

Explore similar work

ResWM: Residual-Action World Model for Visual RL

World Dreamer v5-lite: A Multi-Robot World Action Model with One Dreamer

Preserving Unstable Modes Through Inverse Dynamics in JEPA World Models