Where Should Models Run? A Collaborative Inference Framework for Embodied Intelligence in 6G
Authors: Wanlu Sun, Zhenhui Yuan, Xiang Ji, Yi Han, Qing Li, Gan Zheng
Organizations: School of Engineering, University of Warwick, UK. · School of Information Engineering, Nanchang University, Nanchang, China. · School of Information Engineering, Wuhan University of Technology, Wuhan, China. · Pengcheng Laboratory, Shenzhen, China
Embodied intelligence systems rely on fast and reliable inference to enable real-time perception, reasoning, and control. Recent advances in large-scale artificial intelligence (AI) models, including Large Language Models (LLMs) and Vision-Language-Action (VLA) models, have significantly enhanced the capabilities of embodied agents in complex environments. However, deploying these models remains challenging due to the limited computational resources and energy budgets of robotic platforms. A fundamental question is where inference should be executed. Collaborative inference, which partitions models across end devices, edge servers, and the cloud, has emerged as a promising solution. Yet determining the optimal partitioning point remains an open problem, as it depends on wireless network conditions, model structure, and privacy requirements. This article reviews the state of the art in collaborative inference for embodied intelligence and proposes an edge-centric framework in which the edge coordinates model partitioning and execution. Through both simulation and real-world experiments over WiFi and 5G networks, we demonstrate that the optimal partitioning strategy varies with network conditions and model structure. We further identify latency-minimizing partition points for ResNet-101, Qwen2.5-VL-7B, and OpenVLA-7B in collaborative inference between NVIDIA Jetson Nano or AGX Xavier devices and an AGX Thor edge server under varying wireless conditions. Finally, we discuss the future directions for collaborative inference in 6G-empowered embodied intelligence systems.
Figures & tables
Model
Model type
Organization
Depth a
Params.
Model size (MB)
Deployment
Comm. b latency (ms)
Download latency (ms)
Application
T5
Transformer
Google
E:6–24 D:6–24
60M–11B
240–44,000
Edge/Cloud
–
1000
Language processing
Qwen2.5-VL-7B
Transformer
Alibaba
ViT: 32 LLM: 28
8.29B
33,169
Device/Edge
–
1000
Multimodal reasoning
NoMaD
Diffusion policy
UC Berkeley
4 (Trans.)
19M
76
Device/Edge
12 (DL)
1000
Navigation and exploration
MobileNetV3
CNN
Google
11/15 (blocks)
2.54M–5.48M
10.2–21.9
Device/Edge
15 (UL)
1000
Object recognition
ResNet-101
CNN
Microsoft Research
101 (conv./FC)
44.5M
178
Edge/Cloud
15 (UL)
1000
Image recognition
YOLO26n
CNN
Ultralytics
260 (modules)
2.57M
10.3
Device
–
1000
Object detection
Table 1: Representative DNN models for embodied intelligence and their communication KPIs and place of deployment in alignment with 3GPP TS 22.261 [ 2 ] .
Figure 1: Illustrative architecture of collaborative inference across device, edge and cloud aligned with ITU-T Y.4509.
Figure 2: Functional architecture for edge-centric device–edge collaborative inference, informed by ITU-T Y.4509 and 3GPP TR 37.817. The edge server determines the model partition point(s) based on device, network, model structure and edge-server conditions. The architecture supports both two-way splitting (device–edge) and three-way splitting (device–edge–device), with the prediction head retained on the device in the latter.
Model
SNR
Partitioning point
Communication latency (ms)
Inference latency at device (ms)
Inference latency at edge (ms)
E2E latency (ms)
ResNet-101
Excellent
k=4
27.69
40.12
3.52
71.33
Good
k=4
66.23
40.12
3.52
109.87
Fair
k=8
162.56
74.37
3.00
239.93
Poor
k=31
207.85
246.92
0.41
455.18
Bad
k=31
307.84
246.92
0.41
555.17
Qwen2.5-VL-7B
Excellent
k=S1
22.97
63.87
433.93
520.77
Table 2: Simulated collaborative inference under five wireless channel conditions. For each model and condition, the table reports the latency-minimizing partition among the evaluated candidates and its latency components.
Figure 3: Collaborative inference testbed comprising a camera-equipped robotic rover with a Jetson AGX Xavier, a private 5G link, and a Jetson AGX Thor edge server. The perception task is to identify a person wearing a black top and carrying a white backpack.
Channel condition
Partitioning point
Communication latency (ms)
Inference latency at device (ms)
Inference latency at edge (ms)
E2E latency (ms)
Inference energy at edge (J)
Target identified
C1
k=S1
344.96
78.02
1910.42
2426.73
30.92
Yes
k=S2
344.65
130.60
1872.93
2447.24
30.74
Yes
k=S3
357.99
187.06
1859.31
2501.45
30.52
Yes
k=S4
340.24
244.65
1841.87
2522.61
30.33
Yes
k=SM
283.66
247.26
1835.07
2461.97
30.34
Yes
C2
k=S1
714.65
79.80
2177.91
3062.52
32.94
Yes
Table 3: Hardware performance of Qwen2.5-VL-7B collaborative inference under three measured wireless channel conditions. Bold values identify the lowest reported E2E latency within each condition.