Video TokenCom: Textual Intent-Guided Multi-Rate Video Token Communications with UEP-Based Adaptive Source-Channel Coding
Authors: Jingxuan Men, Mahdi Boloursaz Mashhadi, Ning Wang, Yi Ma, Mike Nilsson, Rahim Tafazolli
Organizations: 5GIC & 6GIC, Institute for Communication Systems (ICS), University of Surrey, U.K. · Smart Internet Lab, University of Bristol, U.K. · British Telecom Research Lab, BT Group plc, U.K.
Token Communication (TokenCom) is a new paradigm, motivated by the recent success of Large AI Models (LAMs) and Multimodal Large Language Models (MLLMs), where tokens serve as unified units of communication and computation, enabling efficient semantic- and goal-oriented information exchange in future wireless networks. In this paper, we propose a novel Video TokenCom framework for textual intent-guided multi-rate video communication with Unequal Error Protection (UEP)-based source-channel coding adaptation. The proposed framework integrates user-intended textual descriptions with discrete video tokenization and unequal error protection to enhance semantic fidelity under restrictive bandwidth constraints. First, discrete video tokens are extracted through a pretrained video tokenizer, while text-conditioned vision-language modeling and optical-flow propagation are jointly used to identify tokens that correspond to user-intended semantics across space and time. Next, we introduce a semantic-aware multi-rate bit-allocation strategy, in which tokens highly related to the user intent are encoded using full codebook precision, whereas non-intended tokens are represented through reduced codebook precision differential encoding, enabling rate savings while preserving semantic quality. Finally, a source and channel coding adaptation scheme is developed to adapt bit allocation and channel coding to varying resources and link conditions. Experiments on various video datasets demonstrate that the proposed framework outperforms both conventional and semantic communication baselines, in perceptual and semantic quality on a wide SNR range.
Figures & tables
Fig. 1: Overall architecture of the proposed Video Token Communications framework with Multi-rate Textual Intent Source-Channel Coding Adaptation. The pipeline consists of: (1) proposed token-based textual intent-guided source encoder with multimodal user-intended token extraction and semantic-aware multi-rate bit coding; (2) proposed token-based source decoder; and (3) proposed UEP-based source channel coding/decoding adaptation.
Algorithm 1 Token-Level Semantic Extraction and Mask Generation
Eref=Zref.
Algorithm 2 Semantic-aware Differential Token Encoding and Reconstruction
pk(c)=BLER(mk(c),rk(c)∣γ).
Algorithm 3 Joint Bit-Precision and MCS Adaptation
UVG Video Sample
PSNR
LPIPS
FVD
TokenCom
VC-DM [ 25 ]
H.265 [ 24 ]
TokenCom
VC-DM [ 25 ]
H.265 [ 24 ]
TokenCom
VC-DM [ 25 ]
H.265 [ 24 ]
UVG-ShakeNDry
26.06
24.68
24.43
0.1296
0.111
0.158
1076
1400
2896
UVG-Beauty
30.57
28.45
25.28
0.0469
0.057
0.17
554
1416
3227
UVG-Bosphorus
28.94
24.55
26.70
0.0863
0.101
0.104
1406
2079
2951
UVG-HoneyBee
22.49
—
21.55
0.1683
—
0.196
504
—
1446
UVG-Jockey
24.31
22.82
20.95
0.0933
0.147
0.201
965
2349
6426
TABLE I: Performance comparison between the proposed video TokenCom framework (BPP=0.013), with VC-DM [ 25 ] and H.265 [ 24 ] (BPP=0.02) baselines on the UVG dataset.
Fig. 3: Performance of the proposed video TokenCom framework with different textual intents: “The woman is hitting the man’s mobile phone.” and “Sky.”. The bit-precision for transmitting user intended and non-intended tokens is 16 and 11 bits per token, respectively. The red rectangle shows the user-intended regions guided by the textual intent.
Fig. 4: Performance of the proposed video TokenCom framework. The red rectangle shows the user-intended regions guided by the textual intent “Car and person.”
Fig. 5: The comparison results with benchmark under the MCL-JCV (a) and UVG (b) datasets. Note that some H.265 points are not shown at low SNRs because the adaptive H.265 pipeline frequently “Failed” to decode more than 85% of the frames, and such cases are marked as invalid in our evaluation.
QP-based constant bitrate mode; uniform per frame/slice
Unequal error protection
Semantic UEP (separate MCS for token classes)
No UEP
TABLE II: Comparison of Key Parameters Between the Proposed video TokenCom Framework and H.265
Fig. 6: (a) The achievable PSNR surface for source and channel coding distortion optimization; (b) The achievable delay surface for source and channel coding adaptation, by the proposed video TokenCom framework.
(a) TokenCom (Proposed) SNR Mod int R int Mod non R non BΔ Source Bitrate PSNR -2 QPSK 0.333 QPSK 0.500 12 165 26.69 0 QPSK 0.333 QPSK 0.500 12 165 27.40 2 QPSK 0.333 QPSK 0.500 12 165 27.40 4 QPSK 0.500 QPSK 0.500 16 173 28.82 6 QPSK 0.500 16QAM 0.500 16 173 28.82 8 QPSK 0.500 16QAM 0.500 16 173 28.82
TABLE III: Side-by-Side Comparison of the proposed video TokenCom framework and H.265 Baseline for a typical video. (“Failed” means that less than 85% of the frames could be reconstructed successfully.)
Fig. 7: The ablation results for the proposed video TokenCom framework with and without textual intent guidance on the MCL-JCV dataset.
School of Information and Electronics, Beijing Institute of Technology (Zhuhai), Zhuhai 519088, China · Department of Electrical and Computer Engineering, The University of Hong Kong, Pokfulam Road, Hong Kong SAR · University of Surrey, Guildford, U.K. +1