SCALE: Real-Time Latent Communication for Cyber-Physical Agents

Shanghao Shi; Jian Xiao; Ching-Hsiang Chan; Sanjoy Baruah; Sastry Kompella; Yevgeniy Vorobeychik; William Yeoh; Ning Zhang

cs.RONov 5, 2026

SCALE: Real-Time Latent Communication for Cyber-Physical Agents

Authors: Shanghao Shi, Jian Xiao, Ching-Hsiang Chan, Sanjoy Baruah, Sastry Kompella, Yevgeniy Vorobeychik, William Yeoh, Ning Zhang

Organizations: Washington University in St. Louis · Nexcepta Inc.

Abstract

Latent communication enables agents to collaborate efficiently by directly exchanging intermediate model representations rather than raw sensor data or text messages. Existing approaches, however, are primarily designed for LLM agents and do not consider the bandwidth constraints and heterogeneous task deadlines faced by cyber-physical agents. In this work, we propose SCALE, a real-time latent communication method tailored for cyber-physical agents. Motivated by the spatio-temporal continuity of the physical world, SCALE matches tokens across consecutive latent messages using similarity scores to identify reusable information and avoid redundant transmission. It then profiles the semantic gain and communication cost of transmitting different amounts of latent information. Based on these profiles, SCALE employs a real-time scheduler that jointly determines which pending messages to transmit and how many latent tokens to send, subject to available bandwidth and task deadlines. We evaluate SCALE across three cyber-physical benchmarks and a physical spider robot case study under diverse network conditions and heterogeneous task deadlines. Our results show that SCALE significantly improves both task performance and on-time delivery in bandwidth-constrained networks. Our code is available at: https://anonymous.4open.science/r/SCALE-4E80/.

Figures & tables

Figure 1: Comparison between Scale and other latent communication baselines on the OPV2V benchmark at 15 Mbps. Scale achieves the best mIoU scores and message on-time rates.

Figure 2: Scale system design. Semantic change evaluation and utility–cost profiling guide the real-time scheduler in jointly selecting messages and transmission sizes under bandwidth and deadline constraints.

(a) Detection
Method	15 Mbps		20 Mbps		25 Mbps		30 Mbps		Varying
	AP $\uparrow$	OT $\uparrow$	AP $\uparrow$	OT $\uparrow$	AP $\uparrow$	OT $\uparrow$	AP $\uparrow$	OT $\uparrow$	AP $\uparrow$	OT $\uparrow$
Full-send	22.77	0.00	22.77	0.00	22.77	0.00	22.77	0.00	40.26	49.97
Uniform	22.50	68.42	23.38	68.64	23.72	68.63	24.91	68.50	39.25	81.59
CoSDH	27.99	32.04	31.34	48.02	40.67	68.14	43.33	75.43	49.87	89.05
V2X-ViT	25.24	0.00	25.24	0.00	25.33	1.21	25.91	8.70	42.22	55.01

Table 1: CooperScene results on 3,936 detection jobs and 18,693 prediction targets. AP denotes average precision, OT denotes the on-time rate (%), and Succ. denotes prediction success (%).

(a) Detection
Method	8 Mbps		15 Mbps		31 Mbps		61 Mbps		Varying
	AP $\uparrow$	OT $\uparrow$	AP $\uparrow$	OT $\uparrow$	AP $\uparrow$	OT $\uparrow$	AP $\uparrow$	OT $\uparrow$	AP $\uparrow$	OT $\uparrow$
Full-send	76.37	0.00	76.37	0.00	76.37	0.00	76.37	0.00	76.37	0.00
Uniform	78.29	46.49	79.87	44.21	77.80	41.20	81.90	39.93	79.65	42.30
CooperTrim	84.25	0.73	84.39	4.92	85.81	28.71	89.47	69.55	86.14	31.81
Where2comm	82.25	0.82	82.63	10.30	88.09	54.42	91.88	86.51	86.20	36.37

Table 2: OPV2V results on 528 detection jobs and 265 segmentation jobs. AP denotes average precision, mIoU denotes mean IoU score, and OT denotes on-time rate (%).

Bandwidth	Method	Task success $\uparrow$	On-time $\uparrow$
3.4 Mbps	Full-send	0.00%	0.00%
	Uniform	0.00%	74.47%
	Scale	44.44%	77.76%
5.0 Mbps	Full-send	24.44%	25.24%
	Uniform	0.00%	74.47%
	Scale	86.67%	84.57%

Table 3: LIBERO results over 45 episodes per method and bandwidth setting. We report the task success rate (%) and on-time rate (%).

Figure 3: Message-lateness CDFs for CooperScene at 20 Mbps: (a) object detection and (b) trajectory prediction. The vertical dashed line marks each task’s deadline; the cumulative percentage at zero gives the on-time rate.

Figure 4: Spider robot patrolling in the mini-city with pedestrians, stop signs, and model cars. The spider patrols along an L-shaped route.

		Bandwidth (Mbps)
Method	Metric	1	2	8
RGB	Success $\uparrow$	0.00%	0.00%	100.00%
RGB	On-time $\uparrow$	0.00%	0.00%	100.00%
Latent	Success $\uparrow$	0.00%	0.00%	100.00%
Latent	On-time $\uparrow$	0.00%	0.00%	100.00%
SCALE	Success $\uparrow$	77.62%	91.67%	100.00%

Table 4: Spider robot mini-city case study results under different bandwidth constraints.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

(a) Tasks and timing
Benchmark	Task	Test workload	Rate (Hz)	Deadline (ms)
CooperScene	3D detection	3,936 jobs	10	100
CooperScene	Trajectory prediction (5 s)	18,693 targets	5	200
OPV2V	3D detection	528 jobs	10	100
OPV2V	BEV segmentation	265 jobs	5	200
LIBERO	Object manipulation	45 episodes	20	Variable

Appendix

Table 5: Detailed experiment settings.

Bandwidth	Change evaluation	Utility–cost profiling	State update + scheduling	Total
15 Mbps	0.351	0.130	0.333	0.813
20 Mbps	0.355	0.130	0.374	0.859
25 Mbps	0.353	0.130	0.390	0.873
30 Mbps	0.354	0.130	0.401	0.884
Varying	0.352	0.129	0.392	0.873

Appendix

Table 6: Mean runtime per message of Scale on CooperScene (ms).

Figure 5: LIBERO task execution with SCALE at 10.1 Mbps: the robot picks up the soup can and places it in the basket. The sequence shows the start, midpoint, and successful completion of one test episode.

Figure 6: Vehicle detection on CooperScene at 25 Mbps. (a) Receiver camera. (b–c) Full-send and SCALE detections for the same frame, using the same CoBEVT model. Green boxes are detections; dashed blue boxes are ground truth. Both BEV panels use the same view.

Figure 7: Vehicle detection on OPV2V at 31 Mbps. (a) Receiver camera. (b–c) Full-send and SCALE detections for the same frame, using the same CoBEVT model. Green boxes are detections; dashed blue boxes are ground truth. Both BEV panels use the same view.

Figure 8: CooperScene message-lateness CDFs at the five selected bandwidth settings: detection (left) and prediction (right). Varying switches between 30 and 200 Mbps every 2 seconds. Zero marks each message’s deadline; the CDF at zero is its method’s on-time rate.

Figure 9: OPV2V message-lateness CDFs at the five selected bandwidth settings: detection (left) and segmentation (right). Varying switches between 8 and 61 Mbps every 2 seconds. Zero marks each message’s deadline; the CDF at zero is its method’s on-time rate.

Figure 10: LIBERO camera-update lateness CDFs at the five selected bandwidth settings, using all 45 episodes per method and setting. Varying switches between 3.4 and 20 Mbps every 2 seconds. Zero marks each message’s deadline; the CDF at zero is its method’s on-time rate.

Multi-Agent Communication

Multi-agent communication studies how autonomous agents exchange information to coordinate decisions, share knowledge, and act together under constraints.

See all 388 papers

BANDMAS: Causality-Inspired Semantic Packet Scheduling for Bandwidth-Efficient Multi-Agent Collaboration
Aug 1, 2026Jiangwen Dong, Wanyu LinMulti-Agent LLM Systems Multi-Agent System Optimization
Department of Computing, The Hong Kong Polytechnic University, Hong Kong SAR, China
HyLaT: Efficient Multi-Agent Communication via Hybrid Latent-Text Protocol
May 25, 2026Xinyi Mou, Siyuan Wang, Zejun Li +2Agent Communication Protocols LLM Agent Training
Fudan University · The Chinese University of Hong Kong · King’s College London +2
Communication-Efficient Digital-Twin Coordination for Heterogeneous LLM Embodied Agents over Computing Power Networks
Jul 10, 2026Nuocheng Yang, Sihua Wang, Zihan Chen +2Multi-Agent LLM Systems Multi-Agent Coordination
Beijing Laboratory of Advanced Information Network, and the Beijing Key Laboratory of Network System Architecture and Convergence, Beijing University of Posts and Telecommunications, Beijing 100876, China · Information Systems Technology and Design Pillar, Singapore University of Technology and Design, 487372, Singapore

Aug 1, 2026J/K move · Enter open · S save

BANDMAS: Causality-Inspired Semantic Packet Scheduling for Bandwidth-Efficient Multi-Agent Collaboration

Jiangwen Dong, Wanyu Lin

Department of Computing, The Hong Kong Polytechnic University, Hong Kong SAR, China

Multi-Agent LLM Systems Multi-Agent System Optimization Multi-Agent Collaboration Multi-Agent Communication

Open paper AskPDF arXiv

LLM-based multi-agent systems make decisions based on the aggregated information via exchanging messages across specialized agents. Forwarding every generated message among agents increases application-layer traffic. Yet, it introduces tremendous input tokens for agent processing, potentially raising inference latency and computational overhead. Existing approaches attempt to address the above issues by pruning agents or discarding redundant messages. Nevertheless, such agent-level or message-level optimization results in insufficient evidence supporting for final decisions or still containing redundant message transmissions. To address these challenges, we propose BANDMAS, a multi-agent collaboration framework that models inter-agent communications as task-oriented traffic, which enables efficient transmission via causality-inspired replay valuation. Specifically, we decompose messages into several data packets by analyzing their semantic features such as evidence and requests. The system only transmits these packets if their predicted replay-derived contribution exceeds their resource cost. Consequently, BANDMAS is able to adaptively schedule communication packets while adhering to bandwidth, latency, deadline, and receiver context constraints. On frozen Qwen3-4B traffic across SciFact, HotpotQA, and FanOutQA, our framework reduces application-layer bytes by 53.2% to 77.3% at selected caps and attains the highest mean task metric among constrained methods on all three workloads.

Abstract

Multi-Agent Communication

Explore similar work

BANDMAS: Causality-Inspired Semantic Packet Scheduling for Bandwidth-Efficient Multi-Agent Collaboration

HyLaT: Efficient Multi-Agent Communication via Hybrid Latent-Text Protocol

Communication-Efficient Digital-Twin Coordination for Heterogeneous LLM Embodied Agents over Computing Power Networks