Video TokenCom: Textual Intent-Guided Multi-Rate Video Token Communications with UEP-Based Adaptive Source-Channel Coding
Authors: Jingxuan Men, Mahdi Boloursaz Mashhadi, Ning Wang, Yi Ma, Mike Nilsson, Rahim Tafazolli
Organizations: 5GIC & 6GIC, Institute for Communication Systems (ICS), University of Surrey, U.K. · Smart Internet Lab, University of Bristol, U.K. · British Telecom Research Lab, BT Group plc, U.K.
Token Communication (TokenCom) is a new paradigm, motivated by the recent success of Large AI Models (LAMs) and Multimodal Large Language Models (MLLMs), where tokens serve as unified units of communication and computation, enabling efficient semantic- and goal-oriented information exchange in future wireless networks. In this paper, we propose a novel Video TokenCom framework for textual intent-guided multi-rate video communication with Unequal Error Protection (UEP)-based source-channel coding adaptation. The proposed framework integrates user-intended textual descriptions with discrete video tokenization and unequal error protection to enhance semantic fidelity under restrictive bandwidth constraints. First, discrete video tokens are extracted through a pretrained video tokenizer, while text-conditioned vision-language modeling and optical-flow propagation are jointly used to identify tokens that correspond to user-intended semantics across space and time. Next, we introduce a semantic-aware multi-rate bit-allocation strategy, in which tokens highly related to the user intent are encoded using full codebook precision, whereas non-intended tokens are represented through reduced codebook precision differential encoding, enabling rate savings while preserving semantic quality. Finally, a source and channel coding adaptation scheme is developed to adapt bit allocation and channel coding to varying resources and link conditions. Experiments on various video datasets demonstrate that the proposed framework outperforms both conventional and semantic communication baselines, in perceptual and semantic quality on a wide SNR range.
Figures & tables
Fig. 1: Overall architecture of the proposed Video Token Communications framework with Multi-rate Textual Intent Source-Channel Coding Adaptation. The pipeline consists of: (1) proposed token-based textual intent-guided source encoder with multimodal user-intended token extraction and semantic-aware multi-rate bit coding; (2) proposed token-based source decoder; and (3) proposed UEP-based source channel coding/decoding adaptation.
Algorithm 1 Token-Level Semantic Extraction and Mask Generation
Eref=Zref.
Algorithm 2 Semantic-aware Differential Token Encoding and Reconstruction
pk(c)=BLER(mk(c),rk(c)∣γ).
Algorithm 3 Joint Bit-Precision and MCS Adaptation
UVG Video Sample
PSNR
LPIPS
FVD
TokenCom
VC-DM [ 25 ]
H.265 [ 24 ]
TokenCom
VC-DM [ 25 ]
H.265 [ 24 ]
TokenCom
VC-DM [ 25 ]
H.265 [ 24 ]
UVG-ShakeNDry
26.06
24.68
24.43
0.1296
0.111
0.158
1076
1400
2896
UVG-Beauty
30.57
28.45
25.28
0.0469
0.057
0.17
554
1416
3227
UVG-Bosphorus
28.94
24.55
26.70
0.0863
0.101
0.104
1406
2079
2951
UVG-HoneyBee
22.49
—
21.55
0.1683
—
0.196
504
—
1446
UVG-Jockey
24.31
22.82
20.95
0.0933
0.147
0.201
965
2349
6426
TABLE I: Performance comparison between the proposed video TokenCom framework (BPP=0.013), with VC-DM [ 25 ] and H.265 [ 24 ] (BPP=0.02) baselines on the UVG dataset.
Fig. 3: Performance of the proposed video TokenCom framework with different textual intents: “The woman is hitting the man’s mobile phone.” and “Sky.”. The bit-precision for transmitting user intended and non-intended tokens is 16 and 11 bits per token, respectively. The red rectangle shows the user-intended regions guided by the textual intent.
Fig. 4: Performance of the proposed video TokenCom framework. The red rectangle shows the user-intended regions guided by the textual intent “Car and person.”
Fig. 5: The comparison results with benchmark under the MCL-JCV (a) and UVG (b) datasets. Note that some H.265 points are not shown at low SNRs because the adaptive H.265 pipeline frequently “Failed” to decode more than 85% of the frames, and such cases are marked as invalid in our evaluation.
QP-based constant bitrate mode; uniform per frame/slice
Unequal error protection
Semantic UEP (separate MCS for token classes)
No UEP
TABLE II: Comparison of Key Parameters Between the Proposed video TokenCom Framework and H.265
Fig. 6: (a) The achievable PSNR surface for source and channel coding distortion optimization; (b) The achievable delay surface for source and channel coding adaptation, by the proposed video TokenCom framework.
(a) TokenCom (Proposed) SNR Mod int R int Mod non R non BΔ Source Bitrate PSNR -2 QPSK 0.333 QPSK 0.500 12 165 26.69 0 QPSK 0.333 QPSK 0.500 12 165 27.40 2 QPSK 0.333 QPSK 0.500 12 165 27.40 4 QPSK 0.500 QPSK 0.500 16 173 28.82 6 QPSK 0.500 16QAM 0.500 16 173 28.82 8 QPSK 0.500 16QAM 0.500 16 173 28.82
TABLE III: Side-by-Side Comparison of the proposed video TokenCom framework and H.265 Baseline for a typical video. (“Failed” means that less than 85% of the frames could be reconstructed successfully.)
Fig. 7: The ablation results for the proposed video TokenCom framework with and without textual intent guidance on the MCL-JCV dataset.
Token Communications (TokenCom) has recently emerged as a new paradigm in which tokens serve as unified units for communication and computation, enabling efficient multimodal semantic and goal-oriented transmission. In this paper, we develop Ada-TokenCom, a rate-adaptive TokenCom framework based on large autoregressive models, which integrates next-token prediction with arithmetic coding to achieve ultra-low bitrate semantic communication at the token level. We propose a mixed reconstruction/generation scheme, where the transmitter encodes and transmits the highly informative tokens at the beginning of the token sequence leveraging a pre-trained autoregressive large model, while the receiver uses an identical model to predict the rest. Moreover, we design a Lyapunov-based algorithm to dynamically optimize both the source compression rate and the modulation and coding scheme, adapting to time-varying network conditions. Simulation results demonstrate that our proposed Ada-TokenCom framework outperforms both digital and deep joint source-channel coding-based semantic communication baselines.
Zijun Zhang, Li Qiao, Mahdi Boloursaz Mashhadi +3
School of Information and Electronics, Beijing Institute of Technology (Zhuhai), Zhuhai 519088, China · Department of Electrical and Computer Engineering, The University of Hong Kong, Pokfulam Road, Hong Kong SAR · University of Surrey, Guildford, U.K. +1
Tokens have become a unified interface for multimodal foundation models, making visual-token communication a natural paradigm for efficient image delivery. However, existing methods typically rely on static policies that cannot jointly adapt to image content and channel conditions. Moreover, their token-level utility objectives do not necessarily translate into improved image reconstruction quality. In this paper, we propose AdapToC, an adaptive, reconstruction-oriented visual-token communication framework. At the transmitter, an adaptive selector jointly models image content, channel state, and communication budget to perform instance-wise resource allocation. Rather than using a fixed token rate and protection policy, it dynamically determines how many tokens should be transmitted and assigns different protection levels according to token importance and current channel conditions. At the receiver, an adaptive MaskGIT receiver incorporates channel reliability into contextual token modeling. It distinguishes tokens with different reliability levels, preserves high-confidence observations, corrects potentially corrupted tokens, and iteratively reconstructs missing content from the received evidence and global visual context. By co-designing token quantity, unequal protection, and reliability-aware recovery for image-level reconstruction quality, AdapToC achieves a peak mean PSNR gain of 4.20 dB over the strongest static baseline under matched communication costs and state-of-the-art performance among the evaluated visual-token communication methods.
In this paper, we propose a new token compression paradigm for video Multimodal Large Language Models (MLLMs), termed Visual Token Coding (VTC). Inspired by classical video coding principles, e.g., HEVC, VTC performs structured compression by predicting the I/P frames of a video and measuring their frame-wise residuals to estimate token redundancy. Based on this baseline framework, we also enhance VTC with a set of novel dynamic designs, such as Dynamic Resolution Input (DyRSO), Dynamic Token Allocation (DyTA), and Spatial Coverage Top-K (SC-TopK), and term this new approach VTCDy. To validate VTC, we apply it to three MLLMs and conduct experiments on multiple video understanding benchmarks. The experimental results show that VTCDy achieves an average performance retention of 100.1% with a 50% token budget for Qwen3-VL, while still retaining 97.8% of the average performance when the token budget is reduced to 25%. Moreover, as a plug-and-play design, VTC requires no additional tuning of MLLMs for token coding. Our code is available at https://github.com/Msr233/VTC.
Chenxin Fang, Tao Chen, JunChao You +3
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China