Organizations: School of Artificial Intelligence, Nanyang Normal University, Nanyang, China · School of Computer and Information Engineering, Tianjin Normal University, Tianjin, China · School of Information Science and Engineering, Linyi University, Linyi 276000, China
Generative image communication transmits compact semantic tokens under a limited packet budget, where token selection directly affects the final reconstruction quality after the complete packet is decoded. However, accurately estimating the terminal value of every candidate token requires repeated receiver-side reconstruction, resulting in substantial encoder-side computation. To address this problem, we propose ACV-Gate, an adaptive candidate evaluation framework that learns to approximate full-budget counterfactual evaluation and selectively assigns exact evaluations to the most informative candidates. Specifically, a set-aware student is trained using terminal advantages and regrets to predict candidate rankings directly, while a selective refinement mechanism evaluates only a bounded candidate set containing both Local-MDL and direct actions; cost-based thresholds further enable explicit control of the average evaluation workload. Experiments on CIFAR-10 show that ACV-Gate consistently improves reconstruction quality while substantially reducing candidate evaluations; at 0.20 bpp, the primary adaptive configuration improves PSNR over LocalMDL by 0.636 dB with only 2.13 candidate evaluations per image, corresponding to 27.60% of the calls required by the Exact-Full expert. Matched-candidate comparisons, synchronized GPU measurements, and evaluations on STL-10 and 384 *384 scale transfer further demonstrate consistent quality computation trade-offs, with particularly pronounced gains at low bit rates. These results show that combining terminal-value learning with selective candidate evaluation provides an effective and controllable mechanism for allocating encoder computation in packet-constrained generative image communication.
Figures & tables
Fig. 1: ACV-Gate architecture. (a) Offline full-budget rollouts provide candidate advantages and regrets for student and allocation-score learning. (b) At the initial communication state, the student ranks the compact proposal. The allocation score and remaining candidate budget select either the direct action or a screened exact evaluation. (c) The selected action is followed by Local-MDL continuation, packet transmission and receiver reconstruction. Dashed arrows indicate transfer of learned parameters and the calibrated threshold.
Fig. 2: Effect of evaluation horizon on candidate selection. Immediate and full-budget selection use the same proposal, state and receiver. Both selected actions are evaluated using the full-budget terminal objective.
Rate
Method
Δ Local (dB)
Ncf /img.
GR (%)
QF (%)
0.20
Local
0.000
0.00
0.00
0.00
Direct ACV-Gate
+0.313 ± 0.088
0.00
23.15
0.00
Adaptive ACV-Gate
+0.636 ± 0.119
2.13 ± 0.53
46.99
27.60
Exact-Full QB expert
+1.353
7.71
100.00
100.00
0.32
Local
0.000
0.00
0.00
0.00
Direct ACV-Gate
+0.020 ± 0.061
0.00
3.75
0.00
TABLE I: End-to-end quality and candidate workload under identical packet and receiver settings. Direct and Adaptive ACV-Gate entries are means ± standard deviations over three selector seeds on the same 200 image-disjoint validation images; Local and Exact-Full QB are fixed 200-image protocol references. Adaptive uses the full proposal on called images ( Cmax=8 ). Ncf counts candidate-level exact calls. Gain recovery (GR) and exact-query fraction (QF) use the corresponding fixed Exact-Full QB row as denominator. Actual bpp is 0.1986, 0.3156 and 0.4375 at the three rates.
Fig. 3: Common-screen reconstruction gain versus candidate workload. The four student families share a four-candidate screening rule and student-margin allocation. Values are mean Δ PSNR relative to Local over three seeds; bars show seed standard deviations. Terminal outcomes are evaluated from cached QB values.
Fig. 4: Paired direct gains for Full and DeepSets-regret-soft on the same validation images. The diagonal denotes equal gain, and the zero axes mark the Local baseline. Annotations report the mean gains.
Fig. 5: Actual-cost threshold transfer for Full ACV-Gate. Thresholds are calibrated on a reserved development monitor at a one-call target and frozen before validation. Error bars are paired image-bootstrap intervals.
Fig. 6: Fixed-screen allocation-score controls. The four practical scores share the same Full direct student, four-candidate screen and target budgets; the post-hoc oracle ranks by the realized V4 screen value and gives the allocation upper bound. Error bars denote seed standard deviations.
Fig. 7: Measured runtime–quality and workload–quality frontiers. Upper panels use synchronized total time; lower panels use realized candidate calls per image. Error bars are paired image-bootstrap intervals. Solid markers are measured branches; dashed curves are B1 Local/Exact-Full and B2 Local/Local-ranked-4 route mixtures over the fixed route seeds. Full and A1 student paths use their frozen seed- 20260817 instances; the repeated five-pass branch measurements are given in Supplementary Table S4.
Fig. 8: Reconstruction gain of adapted token-selection methods and ACV-Gate under the same packet and receiver. Direct and Adaptive are three-seed Full means, with error bars denoting seed standard deviations. The Ncf column in each panel reports mean candidate calls per image.
Fig. 9: Full adaptive ACV-Gate and the fixed Exact-Full QB reference on the independent STL-10 evaluation. Adaptive points are three-seed means with seed standard deviations; the dashed Exact-Full curve is annotated with its 7.78 candidate-level calls per image. The reported paired image intervals for the adaptive branch are positive at all three rates.
Generative visual-token communication reduces transmission load by sending only selected discrete tokens and reconstructing missing content at the receiver. However, existing token-selection criteria based on local uncertainty, importance, or diversity do not directly determine whether changing the current selection improves the final reconstruction under the same packet budget. To address this problem, we propose Gated Counterfactual Refinement for Communication (GCR-C), a rollout-style correction layer over Local-MDL. GCR-C constructs a compact diversified candidate set, evaluates each candidate through matched full-budget Local-MDL continuation, and replaces the baseline action only when a positive baseline-relative reconstruction gain is obtained. Experiments on CIFAR-10, STL-10, a coded 5G-LDPC link, and a limited high-resolution Kodak transfer show that GCR-C consistently improves reconstruction quality at active low- and medium-rate operating points without increasing the realized packet rate, while remaining effective across changes in dataset, channel condition, resolution, token grid, and tokenizer. The results also reveal a clear quality--computation tradeoff due to the additional encoder-side counterfactual evaluation.
Jia Guo, Xiaohan Zhao, Changwang Liu +4
School of Computer and Information Engineering, Tianjin Normal University, Tianjin, China · School of Information Science and Engineering, Linyi University, Linyi 276000, China
Tokens have become a unified interface for multimodal foundation models, making visual-token communication a natural paradigm for efficient image delivery. However, existing methods typically rely on static policies that cannot jointly adapt to image content and channel conditions. Moreover, their token-level utility objectives do not necessarily translate into improved image reconstruction quality. In this paper, we propose AdapToC, an adaptive, reconstruction-oriented visual-token communication framework. At the transmitter, an adaptive selector jointly models image content, channel state, and communication budget to perform instance-wise resource allocation. Rather than using a fixed token rate and protection policy, it dynamically determines how many tokens should be transmitted and assigns different protection levels according to token importance and current channel conditions. At the receiver, an adaptive MaskGIT receiver incorporates channel reliability into contextual token modeling. It distinguishes tokens with different reliability levels, preserves high-confidence observations, corrects potentially corrupted tokens, and iteratively reconstructs missing content from the received evidence and global visual context. By co-designing token quantity, unequal protection, and reliability-aware recovery for image-level reconstruction quality, AdapToC achieves a peak mean PSNR gain of 4.20 dB over the strongest static baseline under matched communication costs and state-of-the-art performance among the evaluated visual-token communication methods.
Under the AI Flow framework, communication networks distribute intelligence across devices, edge servers, and clouds, and computation at the receiver becomes a resource that can substitute for transmitted bits. Generative video compression (GVC) embodies this exchange by sending compact tokens with ultra-low bitrate and letting a generative decoder synthesize the video, yet how much bandwidth savings a unit of decoder compute actually achieves has never been quantified. To fill this vacancy, we model reconstruction quality as a two-factor power law in data rate and decoder compute, which fits measured DISTS of two GVC decoders with a mean error below 3%, and define the information capacity (IC) as the negative logarithmic slope along an iso-quality contour, namely the fraction of rate saved per fractional increase in compute at identical quality. IC is dimensionless and unit-invariant, thus enabling an architecture-agnostic comparison. It forms a field over the operating plane, locating where additional denoising steps are worth their cost. Across five datasets, the 14B decoder trades more compute for fewer rate about ten times more efficiently than the 1.3B decoder. IC also varies significantly across datasets, indicating imbalanced performance on the rate-compute trade-off in GVC methods.
Cheng Yuan, Jiawei Shao, Xuelong Li
Institute of Artificial Intelligence (TeleAI), China Telecom