Ada-TokenCom: Rate-Adaptive Token Communications via Large-Model-Driven Token Compression and Generation
Authors: Zijun Zhang, Li Qiao, Mahdi Boloursaz Mashhadi, Zhen Gao, Mehdi Bennis, Kaibin Huang
Organizations: School of Information and Electronics, Beijing Institute of Technology (Zhuhai), Zhuhai 519088, China · Department of Electrical and Computer Engineering, The University of Hong Kong, Pokfulam Road, Hong Kong SAR · University of Surrey, Guildford, U.K. · Centre for Wireless Communications, University of Oulu, 90014 Oulu, Finland
Token Communications (TokenCom) has recently emerged as a new paradigm in which tokens serve as unified units for communication and computation, enabling efficient multimodal semantic and goal-oriented transmission. In this paper, we develop Ada-TokenCom, a rate-adaptive TokenCom framework based on large autoregressive models, which integrates next-token prediction with arithmetic coding to achieve ultra-low bitrate semantic communication at the token level. We propose a mixed reconstruction/generation scheme, where the transmitter encodes and transmits the highly informative tokens at the beginning of the token sequence leveraging a pre-trained autoregressive large model, while the receiver uses an identical model to predict the rest. Moreover, we design a Lyapunov-based algorithm to dynamically optimize both the source compression rate and the modulation and coding scheme, adapting to time-varying network conditions. Simulation results demonstrate that our proposed Ada-TokenCom framework outperforms both digital and deep joint source-channel coding-based semantic communication baselines.
Figures & tables
Fig. 1: Block diagram of the proposed Ada-TokenCom framework.
Fig. 2: Self-information as a function of token position (a) with class label as conditional information ( cond ) and (b) with null label ( uncond ), averaged over 1000 images from ImageNet-V2 datatset, for pretrained AR models with different parameter scales.
Fig. 3: Visualization of self-information on a 16×16 VQGAN latent grid using LlamaGen-L. Fluctuations predominantly occur at positions associated with discontinuities, which are induced by flattening the 2D latent grid and mapping the grid into 1D token sequence.
Fig. 4: Visualization for (a) heatmap of conditional entropy on a 16×16 VQGAN latent grid, and (b) samples of their decoded image with specified lengths of head tokens.
Model
Params 1
Inference Cost 2
Avg. Bits (cond)
Avg. Bits (uncond)
LlamaGen-B
72 M + 111 M
52.56 G
11.42
11.54
LlamaGen-L
72 M + 343 M
173.57 G
11.22
11.33
Taming Transf.
76 M + 1411 M
364.00 G
8.09
8.37
TABLE I: Average conditional self-information per token under different AR models.
Fig. 5: Performance comparison between the proposed Ada-TokenCom framework and benchmark image compression methods on the ImageNet-V2 dataset.
Fig. 6: Visualization of image reconstruction at different token truncating levels for t2i and c2i scenarios.
QPSK
16QAM
Code
Rate
Polynominal
αm
βm
αm
βm
Uncoded
1
−
1.742
-2.188
8.662
-13.306
Conv Code
1/2
[5,7]8
0.320
0.011
1.103
-0.277
Conv Code
2/3
[51,31,13]8
0.157
0.225
0.506
0.623
Conv Code
1/5
[35, 33, 27, 37, 23] 8
0.086
0.070
0.268
0.075
Turbo Code
1/3
[1,5/7,5/7]8
0.031
0.560
0.087
1.630
TABLE II: Candidate Sets of Channel Coding and Modulations Schemes
n∗
256
224
192
160
128
96
64
32
A
0.937
0.931
0.922
0.913
0.898
0.880
0.861
0.843
TABLE III: Semantic quality A (CLIP) Under Different Prefix Lengths n∗
Fig. 7: Performance comparison of reconstruction among the proposed LlamaGen-based Ada-TokenCom framework, SwinJSCC with rate and SNR adaptation on the ImageNet-V2 dataset.
Fig. 8: Visualization of image reconstruction at different channel bandwidth ratios for the Ada-TokenCom framework and SwinJSCC with rate and SNR adaptation at SNR=12dB . The value shown on each image denotes CBR / CLIP score.
Fig. 9: Effect of the Lyapunov weight η on terminal budget utilization at (a) Cth=500 and (b) Cth=1500 under four SNR regimes.
Fig. 10: Per-slot cumulative CLIP and (1−LPIPS) (upper row) with their historical average link utilization (lower row) under SNR∼N(12,10) when Cth=500 .
Method
Parameters / M (Encoder + Decoder)
Inference Cost / GFlops (Encoder + Decoder)
Encoder Latency
Decoder Latency
SwinJSCC
18.34 + 14.73
35.01 + 34.11
10 ms
6 ms
HiFiC
24.69 + 168.28
15.28 + 116.47
83 ms
188 ms
BPG
–
–
307–824 ms
29–43 ms
Ada-TokenCom (LlamaGen-L) 2
372.21 + 385.36
311.41 + 422.47
19 ms
1012 ms
TABLE IV: Comparison of model size, inference cost, and computation latency 1 .
Token Communication (TokenCom) is a new paradigm, motivated by the recent success of Large AI Models (LAMs) and Multimodal Large Language Models (MLLMs), where tokens serve as unified units of communication and computation, enabling efficient semantic- and goal-oriented information exchange in future wireless networks. In this paper, we propose a novel Video TokenCom framework for textual intent-guided multi-rate video communication with Unequal Error Protection (UEP)-based source-channel coding adaptation. The proposed framework integrates user-intended textual descriptions with discrete video tokenization and unequal error protection to enhance semantic fidelity under restrictive bandwidth constraints. First, discrete video tokens are extracted through a pretrained video tokenizer, while text-conditioned vision-language modeling and optical-flow propagation are jointly used to identify tokens that correspond to user-intended semantics across space and time. Next, we introduce a semantic-aware multi-rate bit-allocation strategy, in which tokens highly related to the user intent are encoded using full codebook precision, whereas non-intended tokens are represented through reduced codebook precision differential encoding, enabling rate savings while preserving semantic quality. Finally, a source and channel coding adaptation scheme is developed to adapt bit allocation and channel coding to varying resources and link conditions. Experiments on various video datasets demonstrate that the proposed framework outperforms both conventional and semantic communication baselines, in perceptual and semantic quality on a wide SNR range.
Jingxuan Men, Mahdi Boloursaz Mashhadi, Ning Wang +3
5GIC & 6GIC, Institute for Communication Systems (ICS), University of Surrey, U.K. · Smart Internet Lab, University of Bristol, U.K. · British Telecom Research Lab, BT Group plc, U.K.
In this paper, we introduce token communications (TokCom), a large model-driven framework to leverage cross-modal context information in generative semantic communications (GenSC). TokCom is a new paradigm, motivated by the recent success of generative foundation models and multimodal large language models (GFM/MLLMs), where the communication units are tokens, enabling efficient transformer-based token processing at the transmitter and receiver. In this paper, we introduce the potential opportunities and challenges of leveraging context in GenSC, explore how to integrate GFM/MLLMs-based token processing into semantic communication systems to leverage cross-modal context effectively at affordable complexity, present the key principles for efficient TokCom at various layers in future wireless networks. In a typical image semantic communication setup, we demonstrate a significant improvement of the bandwidth efficiency, achieved by TokCom by leveraging the context information among tokens. Finally, the potential research directions are identified to facilitate adoption of TokCom in future wireless networks.
Li Qiao, Mahdi Boloursaz Mashhadi, Zhen Gao +3
School of Information and Electronics, Beijing Institute of Technology, Beijing 100081, China · 5GIC & 6GIC, Institute for Communication Systems (ICS), University of Surrey, Guildford, United Kingdom · Centre for Wireless Communications, University of Oulu, 90014 Oulu, Finland +1
In generative semantic communication, semantic tokens guide receiver-side generative models to synthesize high-dimensional content. In challenging network environments, however, frequent token erasure distorts the conveyed semantics beyond what receiver-side recovery can restore. In this paper, we propose a token encoding framework (TokCode) for robust semantic recovery, achieving erasure resilience by restructuring redundancy in the semantic domain. TokCode uses a lightweight adapter to recast a general-purpose large language model (LLM) at the transmitter into a token encoder, exploiting the LLM's pretrained semantic prior to avoid introducing a dedicated deep model. To optimize the adapter efficiently and make it applicable across diverse channels, we develop a channel-quality-aware distillation approach for token encoder training~(CADET). Using a differentiable sentence-level semantic surrogate, CADET tunes T5 foundation models into experts for distinct erasure rates and distills them into a single reconfigurable low-rank adapter, enabling subsequent reinforcement learning (RL) to start above the plateau where direct RL stalls. Simulation results on token-based generative image transmission show that TokCode improves the image-level similarity over the best-performing receiver-side recovery benchmark by 14.1%--22.4%, closing 71.9%--76.5% of its gap to the erasure-aware oracle encoding, when only 20% to 50% of the tokens survive.
Jingzhi Hu, Ouya Wang, Geoffrey Ye Li
Department of Electrical and Electronic Engineering, Imperial College London, London SW7 2AZ, UK.