Training-Free Adaptive 360-degree Video Streaming via Semantic Potential Fields

Aizierjiang Aiersilan; Zhangfei Yang

cs.NIMar 22, 2026

Training-Free Adaptive 360-degree Video Streaming via Semantic Potential Fields

Authors: Aizierjiang Aiersilan, Zhangfei Yang

Organizations: The George Washington University

Abstract

Adaptive 360° video streaming for teleoperation faces two coupled challenges: viewport prediction under uncertain gaze patterns and bitrate adaptation over fluctuating wireless channels. While Deep Reinforcement Learning (DRL) methods achieve high Quality of Experience (QoE), their lack of interpretability and dependence on offline training limit deployment in safety-critical systems. We propose OrbitStream, a training-free framework that formulates viewport prediction as a Gravitational Viewport Prediction (GVP) problem, where semantic objects generate potential fields that attract operator gaze, and employs a Saturation-Based Proportional-Derivative (PD) Controller for buffer regulation. On object-rich teleoperation traces, OrbitStream achieves 94.7% zero-shot viewport prediction accuracy without user-specific profiling, approaching trajectory-extrapolation baselines (~98.5%). Across 3,600 Monte Carlo simulations, it ranks second among 12 algorithms (QoE 2.71 vs. BOLA-E's 2.80), outperforming FastMPC (1.84), with 1.01 ms decision latency and minimal rebuffering.

Figures & tables

Panel of Figure 1 — Fig. 1: Viewport inefficiency in 360° video streaming (top left) and the gravitational potential field in yaw-pitch space. Two potential wells (basins of attraction) correspond to a Vehicle ( $M{=}0.8$ ) and a Pedestrian ( $M{=}1.0$ ); arrows show the gradient field pointing inward. The red trajectory traces gaze from the bottom left, curving past the vehicle and converging into the pedestrian’s potential well, illustrating the physics-based prediction model.

Panel of Figure 2 — Fig. 2: Nonlinear PD buffer controller. The control loop computes the buffer error $e(t)=B(t)-B_{\text{ref}}$ . The PD controller generates a correction signal $u(t)$ , shaped by the smooth nonlinear map $1+\tanh(u(t))$ to prevent aggressive bitrate switching. This factor scales the estimated throughput $\hat{C}(t)$ to derive the target bitrate $R^{*}(t)$ , enabling stable streaming under fluctuating network conditions.

Category	Parameter	Value
Video	Resolution (equirectangular)	3840 $\times$ 1920 pixels
	Frame rate	30 fps
	Segment duration	2.0 seconds
	Tile grid	8 $\times$ 4 (32 tiles)
	Quality levels	6 (1.2M-40.1M bps)
Network	Trace sources	HSDPA, Pensieve, WebRTC

TABLE I: Experimental Configuration

Fig. 3: QoE comparison across twelve algorithms ( OrbitStream is highlighted). Bars report mean QoE with error bars showing

\pm

1 standard deviation.

Fig. 4: Cumulative distribution function (CDF) of average raw QoE across twelve algorithms. The curves highlight differences in tail behavior: aggressive methods exhibit heavier low-QoE tails, while conservative methods show more concentrated distributions.

Algorithm	Mean Buf. (s)	Std Dev (s)	Min Buf. (s)	Switches
Buffer-Based	9.78	0.82	5.45	2.74
Rate-Based	9.72	0.99	4.98	6.27
Pano	9.67	1.04	4.94	5.96
FLARE	9.64	1.06	4.97	12.47
MPC-HM	9.13	0.81	5.84	6.08
MPC	9.10	1.76	4.26	8.78

TABLE II: Buffer Stability and Switching Dynamics (300 Monte Carlo runs per algorithm).

Algorithm	QoE (Raw)	Eqv. Bitrate	Mean Buffer	Decision Time
	(Mean $\pm$ Std)	(Mbps)	(s)	(ms)
BOLA-E	2.80 $\pm$ 0.25	37.49	5.42	0.006
OrbitStream	2.71 $\pm$ 0.31	31.52	5.85	1.010
FastMPC	1.84 $\pm$ 0.49	26.61	5.73	0.019
MPC-HM	1.67 $\pm$ 0.48	22.92	9.13	0.030
Robust-MPC	1.65 $\pm$ 0.60	17.11	7.31	0.033

TABLE III: Performance Summary Across 12 Algorithms (300 Monte Carlo Runs per Algorithm, Raw QoE).

Gaze Estimation

Gaze estimation predicts where a person is looking from eye, face, head-pose, and scene cues.

See all 66 papers

StreamOPD: A Post-Training Recipe with Spatio-Temporal Cue Gating for Streaming Video Understanding
Aug 17, 2026Keming Wu, Baoyi Wang, Kaichen Zhang +7Streaming Video Understanding Streaming Inference
Tsinghua University · Zhejiang University · The University of Hong Kong +3
PAS3R: Pose-Adaptive Streaming 3D Reconstruction for Long Video Sequences
Mar 22, 2026Lanbo Xu, Liang Guo, Caigui Jiang +1Streaming 3D Reconstruction Monocular 3D Reconstruction
State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, China · University of East Anglia, Norwich, NR47TJ, United Kingdom
ScalablePromptus: Scalable and High-Fidelity Prompt-Based Video Streaming
Jul 28, 2026Zehao Cao, Bowei Xu, Xun Cao +2Streaming Video Generation Neural Video Compression
1Nanjing University

Aug 17, 2026J/K move · Enter open · S save

StreamOPD: A Post-Training Recipe with Spatio-Temporal Cue Gating for Streaming Video Understanding

Keming Wu, Baoyi Wang, Kaichen Zhang, Xiang An+6

Tsinghua University · Zhejiang University · The University of Hong Kong+3

Streaming Video Understanding Streaming Inference Temporal Video Understanding Knowledge Distillation On-Policy Distillation

Open paper AskPDF arXiv

Streaming video understanding demands direct responses from the causally observed prefix of an unfolding video. Existing systems add inference-time memory, retrieval, and compression, yet a training-free sliding-window baseline already matches them. We therefore fix a memory-free recent-window protocol and ask how far post-training alone can go. Reinforcement learning with verifiable rewards fits this regime poorly, encouraging long ``think-then-answer'' generations, while on-policy distillation (OPD) supplies dense token-level teacher supervision on student trajectories but is stable only when both models train in thinking mode. These observations lead to StreamOPD, a recipe combining verifiable streaming-video data, thinking-mode OPD, and instruct-mode deployment. It raises StreamingBench from

77.9\%

83.9\%

---within

0.3

points of the 9B teacher---and improves OVO-Bench excluding its hallucination-detection subtask (HLD) by

9.1

points under unchanged inference. As a teacher-privilege extension, Spatio-Temporal CueGate (ST-CueGate) aggregates cue-versus-no-cue teacher likelihood ratios into a group-relative response score that reweights OPD. It reaches

71.9\%

on OVO-Bench (excluding HLD) and

64.9\%

on Video-MME, and is the only variant that stays above the base model on all four benchmarks. Replacing the teacher with a frozen copy of the student's initial policy---on-policy self-distillation---retains most of these gains and lifts HLD to

57.0\%

, above both the untrained student and the 9B teacher, so abstention loss is not intrinsic to the recipe. We provide a transparent and reproducible reference for open-source streaming-video research.

Abstract

Gaze Estimation

Explore similar work

StreamOPD: A Post-Training Recipe with Spatio-Temporal Cue Gating for Streaming Video Understanding

PAS3R: Pose-Adaptive Streaming 3D Reconstruction for Long Video Sequences

ScalablePromptus: Scalable and High-Fidelity Prompt-Based Video Streaming