Selective Amortization of Full-Budget Counterfactual Reasoning for Visual Token Communication

Qinglei Qi; Zhihe Liang; Fengzhan Jing; Shenao Zhu; Lei Zhang; Chenyang Zhang; Shuqing He; Jia Guo

cs.AISep 25, 2026

Selective Amortization of Full-Budget Counterfactual Reasoning for Visual Token Communication

Authors: Qinglei Qi, Zhihe Liang, Fengzhan Jing, Shenao Zhu, Lei Zhang, Chenyang Zhang, Shuqing He, Jia Guo

Organizations: School of Artificial Intelligence, Nanyang Normal University, Nanyang, China · School of Computer and Information Engineering, Tianjin Normal University, Tianjin, China · School of Information Science and Engineering, Linyi University, Linyi 276000, China

Abstract

Generative image communication transmits compact semantic tokens under a limited packet budget, where token selection directly affects the final reconstruction quality after the complete packet is decoded. However, accurately estimating the terminal value of every candidate token requires repeated receiver-side reconstruction, resulting in substantial encoder-side computation. To address this problem, we propose ACV-Gate, an adaptive candidate evaluation framework that learns to approximate full-budget counterfactual evaluation and selectively assigns exact evaluations to the most informative candidates. Specifically, a set-aware student is trained using terminal advantages and regrets to predict candidate rankings directly, while a selective refinement mechanism evaluates only a bounded candidate set containing both Local-MDL and direct actions; cost-based thresholds further enable explicit control of the average evaluation workload. Experiments on CIFAR-10 show that ACV-Gate consistently improves reconstruction quality while substantially reducing candidate evaluations; at 0.20 bpp, the primary adaptive configuration improves PSNR over LocalMDL by 0.636 dB with only 2.13 candidate evaluations per image, corresponding to 27.60% of the calls required by the Exact-Full expert. Matched-candidate comparisons, synchronized GPU measurements, and evaluations on STL-10 and 384 *384 scale transfer further demonstrate consistent quality computation trade-offs, with particularly pronounced gains at low bit rates. These results show that combining terminal-value learning with selective candidate evaluation provides an effective and controllable mechanism for allocating encoder computation in packet-constrained generative image communication.

Figures & tables

Fig. 1: ACV-Gate architecture. (a) Offline full-budget rollouts provide candidate advantages and regrets for student and allocation-score learning. (b) At the initial communication state, the student ranks the compact proposal. The allocation score and remaining candidate budget select either the direct action or a screened exact evaluation. (c) The selected action is followed by Local-MDL continuation, packet transmission and receiver reconstruction. Dashed arrows indicate transfer of learned parameters and the calibrated threshold.

Fig. 2: Effect of evaluation horizon on candidate selection. Immediate and full-budget selection use the same proposal, state and receiver. Both selected actions are evaluated using the full-budget terminal objective.

Rate	Method	$\Delta$ Local (dB)	$N_{\mathrm{cf}}$ /img.	GR (%)	QF (%)
0.20	Local	0.000	0.00	0.00	0.00
	Direct ACV-Gate	+0.313 $\pm$ 0.088	0.00	23.15	0.00
	Adaptive ACV-Gate	+0.636 $\pm$ 0.119	2.13 $\pm$ 0.53	46.99	27.60
	Exact-Full $Q_{B}$ expert	+1.353	7.71	100.00	100.00
0.32	Local	0.000	0.00	0.00	0.00
	Direct ACV-Gate	+0.020 $\pm$ 0.061	0.00	3.75	0.00

TABLE I: End-to-end quality and candidate workload under identical packet and receiver settings. Direct and Adaptive ACV-Gate entries are means

\pm

standard deviations over three selector seeds on the same 200 image-disjoint validation images; Local and Exact-Full

Q_{B}

are fixed 200-image protocol references. Adaptive uses the full proposal on called images (

C_{\max}=8

N_{\mathrm{cf}}

counts candidate-level exact calls. Gain recovery (GR) and exact-query fraction (QF) use the corresponding fixed Exact-Full

Q_{B}

row as denominator. Actual bpp is 0.1986, 0.3156 and 0.4375 at the three rates.

Fig. 3: Common-screen reconstruction gain versus candidate workload. The four student families share a four-candidate screening rule and student-margin allocation. Values are mean

\Delta

PSNR relative to Local over three seeds; bars show seed standard deviations. Terminal outcomes are evaluated from cached

Q_{B}

values.

Fig. 4: Paired direct gains for Full and DeepSets-regret-soft on the same validation images. The diagonal denotes equal gain, and the zero axes mark the Local baseline. Annotations report the mean gains.

Fig. 5: Actual-cost threshold transfer for Full ACV-Gate. Thresholds are calibrated on a reserved development monitor at a one-call target and frozen before validation. Error bars are paired image-bootstrap intervals.

Fig. 6: Fixed-screen allocation-score controls. The four practical scores share the same Full direct student, four-candidate screen and target budgets; the post-hoc oracle ranks by the realized

V_{4}

screen value and gives the allocation upper bound. Error bars denote seed standard deviations.

Fig. 7: Measured runtime–quality and workload–quality frontiers. Upper panels use synchronized total time; lower panels use realized candidate calls per image. Error bars are paired image-bootstrap intervals. Solid markers are measured branches; dashed curves are B1 Local/Exact-Full and B2 Local/Local-ranked-4 route mixtures over the fixed route seeds. Full and A1 student paths use their frozen seed-

20260817

instances; the repeated five-pass branch measurements are given in Supplementary Table S4.

Fig. 8: Reconstruction gain of adapted token-selection methods and ACV-Gate under the same packet and receiver. Direct and Adaptive are three-seed Full means, with error bars denoting seed standard deviations. The

N_{\mathrm{cf}}

column in each panel reports mean candidate calls per image.

Fig. 9: Full adaptive ACV-Gate and the fixed Exact-Full

Q_{B}

reference on the independent STL-10 evaluation. Adaptive points are three-seed means with seed standard deviations; the dashed Exact-Full curve is annotated with its 7.78 candidate-level calls per image. The reported paired image intervals for the adaptive branch are positive at all three rates.

Explore similar work

Baseline-Relative Counterfactual Refinement for Bit-Aware Visual Token Communication
Aug 17, 2026Jia Guo, Xiaohan Zhao, Changwang Liu +4Visual Token Compression Visual Token Pruning
School of Computer and Information Engineering, Tianjin Normal University, Tianjin, China · School of Information Science and Engineering, Linyi University, Linyi 276000, China
Beyond Token Accuracy: Prioritizing What Matters for Visual Reconstruction
Oct 2, 2026Zhicheng Liu, Zhouxiang Zhao, Chenliang Wu +2Image Reconstruction Wireless Resource Allocation
Zhejiang University
Information Capacity of Generative Video Compression: Quantifying the Rate-Compute Exchange at Identical Quality
Sep 23, 2026Cheng Yuan, Jiawei Shao, Xuelong LiRate-Distortion Optimization Neural Video Compression
Institute of Artificial Intelligence (TeleAI), China Telecom

Aug 17, 2026J/K move · Enter open · S save

Baseline-Relative Counterfactual Refinement for Bit-Aware Visual Token Communication

Jia Guo, Xiaohan Zhao, Changwang Liu, Shuqing He+3

School of Computer and Information Engineering, Tianjin Normal University, Tianjin, China · School of Information Science and Engineering, Linyi University, Linyi 276000, China

Visual Token Compression Visual Token Pruning Generative Image Compression

Open paper AskPDF arXiv

Generative visual-token communication reduces transmission load by sending only selected discrete tokens and reconstructing missing content at the receiver. However, existing token-selection criteria based on local uncertainty, importance, or diversity do not directly determine whether changing the current selection improves the final reconstruction under the same packet budget. To address this problem, we propose Gated Counterfactual Refinement for Communication (GCR-C), a rollout-style correction layer over Local-MDL. GCR-C constructs a compact diversified candidate set, evaluates each candidate through matched full-budget Local-MDL continuation, and replaces the baseline action only when a positive baseline-relative reconstruction gain is obtained. Experiments on CIFAR-10, STL-10, a coded 5G-LDPC link, and a limited high-resolution Kodak transfer show that GCR-C consistently improves reconstruction quality at active low- and medium-rate operating points without increasing the realized packet rate, while remaining effective across changes in dataset, channel condition, resolution, token grid, and tokenizer. The results also reveal a clear quality--computation tradeoff due to the additional encoder-side counterfactual evaluation.

Abstract

Explore similar work

Baseline-Relative Counterfactual Refinement for Bit-Aware Visual Token Communication

Beyond Token Accuracy: Prioritizing What Matters for Visual Reconstruction

Information Capacity of Generative Video Compression: Quantifying the Rate-Compute Exchange at Identical Quality