R2E-VID: Two-Stage Robust Routing via Temporal Gating for Elastic Edge-Cloud Video Inference
Authors: Zheming Yang, Lulu Zuo, Shun Lu, Yangyu Zhang, Zhicheng Li, Xiangyang Li, Yang You
Organizations: Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Peng Cheng Laboratory · Beijing Kuaishou Technology Co., Ltd., China · National University of Singapore
With the rapid growth of large-scale video analytics applications, edge-cloud collaborative systems have become the dominant paradigm for real-time inference. However, existing approaches often fail to dynamically adapt to heterogeneous video content and fluctuating resource conditions, resulting in suboptimal routing efficiency and high computational costs. In this paper, we propose R2E-VID, a two-stage robust routing framework via temporal gating for elastic edge-cloud video inference. In the first stage, R2E-VID introduces a temporal gating mechanism that models the temporal consistency and motion dynamics of incoming video streams to predict the optimal routing pattern for each segment. This enables adaptive partitioning of inference workloads between edge and cloud nodes, achieving fine-grained spatiotemporal elasticity. In the second stage, a robust routing optimization module refines the allocation through multi-model adaptation, jointly minimizing inference delay and resource consumption under dynamic network and workload variations. Extensive experiments on public datasets demonstrate that R2E-VID achieves up to 60% reduction in overall cost compared to cloud-centric baselines, and delivers 35-45% lower delay while improving inference accuracy by 2-7% over state-of-the-art edge-cloud solutions.
Figures & tables
Figure 1. The illustration of edge-cloud collaborative architecture for video inference.
Figure 2. The (a), (b), (c), and (d) are the inference results for different resolutions. The (e) and (f) are the results for different models on edge server and cloud server. The normalized delay and energy consumption are multiplied and then summed to obtain the cost.
Figure 3. The workflow of the proposed R2E-VID framework.
Figure 4. The illustration of adaptive edge-cloud collaborative configuration via temporal gating.
Figure 5. The accuracy-cost tradeoff analysis under different datasets.
Detection Objects
Stable Requirements
Fluctuating Requirements
A2
JCAB
RDAP
Sniper
R2E-VID
A2
JCAB
RDAP
Sniper
R2E-VID
Cars
69.54%
66.83%
63.79%
67.26%
69.67%
68.33%
65.13%
62.11%
66.45%
69.54%
Buses
69.73%
66.25%
62.37%
67.11%
69.81%
68.14%
65.18%
61.16%
66.13%
69.16%
Motorcycles
64.06%
61.12%
57.17%
62.21%
64.15%
63.33%
60.04%
55.62%
61.16%
63.77%
Bicycles
63.15%
59.86%
55.84%
61.03%
63.35%
61.16%
59.24%
54.23%
60.03%
62.86%
Persons
66.31%
62.81%
58.76%
64.12%
66.51%
64.43%
62.13%
57.17%
63.24%
65.73%
Table 1. The average accuracy comparison results under the object detection task (Take the average of the COCO dataset and UA-DETRAC dataset).
Methods
Stable Bandwidths
Fluctuating Bandwidths
MIoU
MPA
MIoU
MPA
A2
50.95
79.18
50.69
79.12
JCAB
48.35
77.14
48.07
76.82
RDAP
45.26
71.31
44.81
71.24
Sniper
49.13
77.72
48.82
77.36
R2E-VID
51.16
79.26
51.09
79.17
Table 2. The average accuracy comparison results under semantic segmentation task.
Methods
A2
JCAB
RDAP
Sniper
Ours
COCO
Stable
94%
87%
86%
88%
97%
Fluctuating
90%
83%
81%
85%
96%
UA-DETRAC
Stable
91%
84%
84%
86%
95%
Fluctuating
88%
81%
79%
82%
94%
ADE20K
Stable
89%
80%
79%
81%
92%
Fluctuating
88%
75%
74%
78%
91%
Table 3. The success rates for meeting the accuracy requirements of task processing.
Figure 6. The comparative results of different methods under the COCO dataset.
Figure 7. The comparative results of different methods under the UA-DETRAC dataset.
Figure 8. The comparative results of different methods under the ADE20K dataset.
Figure 9. The average cost comparison of different methods under dynamic bandwidths.
Figure 10. The ablation studies of two-stage robust optimization.
Edge-cloud computing distributes data processing and AI workloads between nearby devices and remote cloud services to balance latency, cost, energy, and scale.
Long, continuous video streams are an increasingly critical driver of multimedia intelligence. Existing efforts often handle long videos with a sample-encode-reason approach using large models. However, they overlook a crucial deployment fact: the stream is often produced by computationally constrained devices. This forces an untenable compromise: cloud offloading unlocks strong reasoning but incurs prohibitive bandwidth overhead, while on-device processing remains limited by edge hardware capacity. Therefore, we propose CoVStream, the first edge-cloud collaborative framework for understanding long video streams. The edge node distills raw video streams into compact visual features and semantic captions for transmission to the cloud, minimizing bandwidth costs, while the cloud server integrates this data into an entity graph and global visual context, activating the heavy reasoning model only when a user query arrives. Experiments on VideoMME-Long, LVBench, and RTV-Bench show that CoVStream reduces bandwidth usage by 87.6% while retaining 99.2% of the cloud baseline accuracy on LVBench.
Xu Liu, Guikun Chen, Zihao Yan +2
The State Key Lab of Brain-Machine Intelligence, Zhejiang University · vivo Mobile Communication Co., Ltd., Shenzhen, China.
Edge-cloud inference collaborations are often designed with a routing estimator that decides whether to offload each frame from weak models at the edge to stronger models in the cloud. Existing systems place the routing estimator after the weak detector, so the weak forward pass still runs even on frames that are later offloaded. In this paper, we argue that this weak-conditioned design can be suboptimal when the offload budget varies. First, we present a competitive weak-skipping estimator (0.153 GFLOPs, about 29x lighter than the weak detector at 4.49 GFLOPs) that extracts routing signal from raw pixels, outperforming the common after-weak placement weak-conditioned baselines. Second, we show that neither weak-skipping nor weak-conditioned placement dominates across the full operating curve, and we propose budget-adaptive routing, which selects between them by offload budget via two offline-tuned thresholds. On PASCAL VOC, our budget-adaptive router traces the upper accuracy envelope of both fixed placements across the operating range. Our method reduces per-frame latency by up to 19.1 ms (about 30% lower at rho = 0.9). Besides outperforming SOTA methods, it is surprisingly stronger than the strong model (+1.7 pp over the strong model's peak mAP) at some operating points with far less compute. Artifacts are available at https://github.com/ViGeng/bgt-ada
Wei Geng, Nitinder Mohan, Jörg Ott
Technical University of Munich Munich, Germany · TU Delft Delft, Netherlands
VLA models have emerged as the mainstream for embodied intelligence, which decouples reasoning and execution via action chunks. To enable real-time deployment of VLA models, existing studies investigate edge-cloud collaborative inference, which pose two challenges to the action chunk mechanism. (1) Edge-cloud collaboration complicates system latency components, hindering the determination of valid ranges for action chunks. (2) Latency fluctuations in cross-edge update links under edge-cloud scenarios make it difficult to schedule replanning requests. To address these, we analyze the key factors affecting action chunk usable ranges and replanning request timing, revealing phase dependence and edge-cloud latency constraints. Based on these insights, we propose ActECC, which enables Action Chunk Temporal Harnessing through kinematics-guided usable range assessment and delay-aware replanning scheduling. Experiments show that ActECC reduces per-action latency by up to 27.4% and cloud calls by up to 45.1%, while lowering average execution error compared with existing edge-cloud collaborative.
Zihao Zheng, Zhihao Mao, Hui Huang +9
School of Computer Science, Peking University, Beijing, China. · School of Artificial Intelligence, Beijing Normal University, Beijing, China. · School of Software Engineering, South China University of Technology, Guangzhou, China.