Organizations: School of Computer and Information Engineering, Tianjin Normal University, Tianjin, China · School of Information Science and Engineering, Linyi University, Linyi 276000, China
Generative visual-token communication reduces transmission load by sending only selected discrete tokens and reconstructing missing content at the receiver. However, existing token-selection criteria based on local uncertainty, importance, or diversity do not directly determine whether changing the current selection improves the final reconstruction under the same packet budget. To address this problem, we propose Gated Counterfactual Refinement for Communication (GCR-C), a rollout-style correction layer over Local-MDL. GCR-C constructs a compact diversified candidate set, evaluates each candidate through matched full-budget Local-MDL continuation, and replaces the baseline action only when a positive baseline-relative reconstruction gain is obtained. Experiments on CIFAR-10, STL-10, a coded 5G-LDPC link, and a limited high-resolution Kodak transfer show that GCR-C consistently improves reconstruction quality at active low- and medium-rate operating points without increasing the realized packet rate, while remaining effective across changes in dataset, channel condition, resolution, token grid, and tokenizer. The results also reveal a clear quality--computation tradeoff due to the additional encoder-side counterfactual evaluation.
Figures & tables
Fig. 1: System architecture of GCR-C. A compact proposal is evaluated by full-budget Local-MDL completion, and an alternative action is accepted only when its baseline-relative advantage is positive. Selected tokens are packetized and transmitted through the coded link; receiver reconstruction is used only for offline evaluation.
Fig. 2: Validation diagnostics: headroom CDF (a), state headroom/P4 regret with image-cluster CIs (b), horizon quality–latency frontier (c), and proposal-source ablation (d).
Fig. 3: Proposal-size quality–compute frontier; K=8 is the final compact operating point.
Nominal bpp
Method
Packet bpp
Tokens
PSNR (dB)
SSIM
Δ PSNR
95% CI
Win rate
0.20
Random
0.1986
11.41
13.215 ± 3.511
.382
+0.798
[+.552,+1.042]
.548
Entropy-greedy
0.1986
12.37
13.319 ± 3.520
.392
+0.902
[+.674,+1.134]
.576
Coverage-greedy
0.1992
11.00
12.889 ± 3.653
.365
+0.472
[+.237,+.706]
.506
Adapted max–min diversity [ 21 ]
0.1984
12.49
11.903 ± 4.006
.333
− 0.514
[ − 0.775, − 0.245]
.368
Local-MDL
0.1985
12.02
12.417 ± 3.864
.395
0
—
—
GCR-C
0.1987
12.01
13.920 ± 3.637
.436
+1.503
[+1.341,+1.670]
.974
TABLE I: Held-out CIFAR-10 results on 500 images. Values are mean ± standard deviation for PSNR and mean SSIM. Random, Entropy-greedy, Coverage-greedy, and Adapted max–min diversity are matched baselines evaluated with the same packet protocol; the difference, CI, and win rate are paired against Local-MDL. At 0.44 bpp, GCR-C is identical to Local-MDL by policy; therefore, the win rate is 0 and the tie rate is 1.00.
Fig. 4: Held-out CIFAR-10 rate–distortion and paired GCR-C–Local-MDL 95% CIs; matched baselines are in Table I .
Rate
Channel
Eb/N0 (H/M/E)
Δ PSNR (H/M/E)
0.20
AWGN
3.0/3.5/4.5
+.946/+1.194/+1.416
0.20
Block-Rayleigh
4.0/8.0/12.5
+.697/+1.071/+1.304
0.32
AWGN
2.5/3.0/3.5
+.378/+.539/+.647
0.32
Block-Rayleigh
4.0/7.5/12.0
+.394/+.450/+.630
TABLE II: Compact frozen-selection coded-link summary on the held-out CIFAR-10 split. Eb/N0 and Δ PSNR are listed as hard/medium/easy triples; all 12 paired 95% CIs are positive. Complete FER, PSNR, and CI records are provided in Supplementary Table S4.
Point
Method
Cap (bits)
Packet bpp
Tokens
PSNR/ SSIM
Δ PSNR
95% CI
Win
Low
Local-MDL
350
.037907
24.912
11.998/.261
–
–
–
Low
GCR-C
350
.037911
24.754
12.620/.275
+.621
[.559,.687]
99.6%
Mid
Local-MDL
650
.070346
65.678
14.533/.358
–
–
–
Mid
GCR-C
650
.070350
65.630
14.738/.364
+.204
[.169,.243]
94.2%
TABLE III: STL-10 replication on a stratified 500-image held-out subset. Packet bpp is recomputed from complete packet fields; Δ PSNR, CI, and win rate are paired GCR-C–Local-MDL differences.
Point
Method
Tok.
Pkt bpp
PSNR/SSIM
Δ PSNR [CI]
HR-Low
Local-MDL
87.708
.014204
9.349/.241
–
GCR-C
87.667
.014201
10.382/.273
+1.033 [.641,1.456]
HR-Mid
Local-MDL
173.500
.025748
10.329/.238
–
GCR-C
173.542
.025746
11.177/.290
+.847 [.369,1.417]
TABLE IV: Kodak-24 high-resolution transfer. Values are means over 24 images; Packet bpp is realized packet rate and the Δ PSNR interval is a paired 10,000-resample image-level bootstrap CI.
Generative image communication transmits compact semantic tokens under a limited packet budget, where token selection directly affects the final reconstruction quality after the complete packet is decoded. However, accurately estimating the terminal value of every candidate token requires repeated receiver-side reconstruction, resulting in substantial encoder-side computation. To address this problem, we propose ACV-Gate, an adaptive candidate evaluation framework that learns to approximate full-budget counterfactual evaluation and selectively assigns exact evaluations to the most informative candidates. Specifically, a set-aware student is trained using terminal advantages and regrets to predict candidate rankings directly, while a selective refinement mechanism evaluates only a bounded candidate set containing both Local-MDL and direct actions; cost-based thresholds further enable explicit control of the average evaluation workload. Experiments on CIFAR-10 show that ACV-Gate consistently improves reconstruction quality while substantially reducing candidate evaluations; at 0.20 bpp, the primary adaptive configuration improves PSNR over LocalMDL by 0.636 dB with only 2.13 candidate evaluations per image, corresponding to 27.60% of the calls required by the Exact-Full expert. Matched-candidate comparisons, synchronized GPU measurements, and evaluations on STL-10 and 384 *384 scale transfer further demonstrate consistent quality computation trade-offs, with particularly pronounced gains at low bit rates. These results show that combining terminal-value learning with selective candidate evaluation provides an effective and controllable mechanism for allocating encoder computation in packet-constrained generative image communication.
Qinglei Qi, Zhihe Liang, Fengzhan Jing +5
School of Artificial Intelligence, Nanyang Normal University, Nanyang, China · School of Computer and Information Engineering, Tianjin Normal University, Tianjin, China · School of Information Science and Engineering, Linyi University, Linyi 276000, China
Tokens have become a unified interface for multimodal foundation models, making visual-token communication a natural paradigm for efficient image delivery. However, existing methods typically rely on static policies that cannot jointly adapt to image content and channel conditions. Moreover, their token-level utility objectives do not necessarily translate into improved image reconstruction quality. In this paper, we propose AdapToC, an adaptive, reconstruction-oriented visual-token communication framework. At the transmitter, an adaptive selector jointly models image content, channel state, and communication budget to perform instance-wise resource allocation. Rather than using a fixed token rate and protection policy, it dynamically determines how many tokens should be transmitted and assigns different protection levels according to token importance and current channel conditions. At the receiver, an adaptive MaskGIT receiver incorporates channel reliability into contextual token modeling. It distinguishes tokens with different reliability levels, preserves high-confidence observations, corrects potentially corrupted tokens, and iteratively reconstructs missing content from the received evidence and global visual context. By co-designing token quantity, unequal protection, and reliability-aware recovery for image-level reconstruction quality, AdapToC achieves a peak mean PSNR gain of 4.20 dB over the strongest static baseline under matched communication costs and state-of-the-art performance among the evaluated visual-token communication methods.
In generative semantic communication, semantic tokens guide receiver-side generative models to synthesize high-dimensional content. In challenging network environments, however, frequent token erasure distorts the conveyed semantics beyond what receiver-side recovery can restore. In this paper, we propose a token encoding framework (TokCode) for robust semantic recovery, achieving erasure resilience by restructuring redundancy in the semantic domain. TokCode uses a lightweight adapter to recast a general-purpose large language model (LLM) at the transmitter into a token encoder, exploiting the LLM's pretrained semantic prior to avoid introducing a dedicated deep model. To optimize the adapter efficiently and make it applicable across diverse channels, we develop a channel-quality-aware distillation approach for token encoder training~(CADET). Using a differentiable sentence-level semantic surrogate, CADET tunes T5 foundation models into experts for distinct erasure rates and distills them into a single reconfigurable low-rank adapter, enabling subsequent reinforcement learning (RL) to start above the plateau where direct RL stalls. Simulation results on token-based generative image transmission show that TokCode improves the image-level similarity over the best-performing receiver-side recovery benchmark by 14.1%--22.4%, closing 71.9%--76.5% of its gap to the erasure-aware oracle encoding, when only 20% to 50% of the tokens survive.
Jingzhi Hu, Ouya Wang, Geoffrey Ye Li
Department of Electrical and Electronic Engineering, Imperial College London, London SW7 2AZ, UK.