Beyond Token Accuracy: Prioritizing What Matters for Visual Reconstruction

Zhicheng Liu; Zhouxiang Zhao; Chenliang Wu; Zhaohui Yang; Zhaoyang Zhang

cs.CVOct 2, 2026

Beyond Token Accuracy: Prioritizing What Matters for Visual Reconstruction

Authors: Zhicheng Liu, Zhouxiang Zhao, Chenliang Wu, Zhaohui Yang, Zhaoyang Zhang

Organizations: Zhejiang University

Abstract

Tokens have become a unified interface for multimodal foundation models, making visual-token communication a natural paradigm for efficient image delivery. However, existing methods typically rely on static policies that cannot jointly adapt to image content and channel conditions. Moreover, their token-level utility objectives do not necessarily translate into improved image reconstruction quality. In this paper, we propose AdapToC, an adaptive, reconstruction-oriented visual-token communication framework. At the transmitter, an adaptive selector jointly models image content, channel state, and communication budget to perform instance-wise resource allocation. Rather than using a fixed token rate and protection policy, it dynamically determines how many tokens should be transmitted and assigns different protection levels according to token importance and current channel conditions. At the receiver, an adaptive MaskGIT receiver incorporates channel reliability into contextual token modeling. It distinguishes tokens with different reliability levels, preserves high-confidence observations, corrects potentially corrupted tokens, and iteratively reconstructs missing content from the received evidence and global visual context. By co-designing token quantity, unequal protection, and reliability-aware recovery for image-level reconstruction quality, AdapToC achieves a peak mean PSNR gain of 4.20 dB over the strongest static baseline under matched communication costs and state-of-the-art performance among the evaluated visual-token communication methods.

Figures & tables

Figure 1: Examples of mismatch between token accuracy and image quality.

Figure 2: Overview of the end-to-end AdapToC framework.

Figure 3: Comparison with digital communication baselines across channel conditions at

B=6052

Figure 4: Qualitative comparison at SNR $=8$ dB and $B=6052$ .

Figure 5: Sender–receiver ablation ablation at

B=6052

Online component	GFLOPs	Share
Sender actor	0.7030	0.116%
Selected digital PHY	0.0158	0.003%
Contextual completion	34.390	5.679%
Terminal VQ decode	570.465	94.202%
Total	605.574	100.000%

Table 1: Online computation breakdown.

Metric	Ref.	$\delta^{\star}$	Worst	$\Delta$	Rel. $\Delta$
PSNR $\uparrow$	15.646	$+2$	15.385	$-0.2610$	$-1.67\%$
SSIM $\uparrow$	0.4130	$+2$	0.4101	$-0.0029$	$-0.70\%$
LPIPS $\downarrow$	0.5346	$-2$	0.5424	$+0.0078$	$+1.46\%$
CLIP $\uparrow$	0.7597	$-2$	0.7539	$-0.0058$	$-0.76\%$

Table 2: Robustness to sender-side SNR-estimation error.

Explore similar work

Aug 17, 2026

Baseline-Relative Counterfactual Refinement for Bit-Aware Visual Token Communication

Generative visual-token communication reduces transmission load by sending only selected discrete tokens and reconstructing missing content at the receiver. However, existing token-selection criteria based on local uncertainty, importance, or diversity do not directly determine whether changing the current selection improves the final reconstruction under the same packet budget. To address this problem, we propose Gated Counterfactual Refinement for Communication (GCR-C), a rollout-style correction layer over Local-MDL. GCR-C constructs a compact diversified candidate set, evaluates each candidate through matched full-budget Local-MDL continuation, and replaces the baseline action only when a positive baseline-relative reconstruction gain is obtained. Experiments on CIFAR-10, STL-10, a coded 5G-LDPC link, and a limited high-resolution Kodak transfer show that GCR-C consistently improves reconstruction quality at active low- and medium-rate operating points without increasing the realized packet rate, while remaining effective across changes in dataset, channel condition, resolution, token grid, and tokenizer. The results also reveal a clear quality--computation tradeoff due to the additional encoder-side counterfactual evaluation.

Jia Guo, Xiaohan Zhao, Changwang Liu +4

School of Computer and Information Engineering, Tianjin Normal University, Tianjin, China · School of Information Science and Engineering, Linyi University, Linyi 276000, China

Visual Token Compression Visual Token Pruning Generative Image Compression

Ask PDF

Oct 9, 2026

MixTok: Adaptive Visual Tokenization with Global 1D Tokens and Sparse 2D Refinement

Efficient visual tokenization requires balancing representation compactness with the preservation of fine-grained image content. Conventional grid-based tokenizers allocate tokens uniformly across spatial locations despite non-uniform visual complexity, whereas compact 1D tokenizers aggregate global information but can struggle to preserve local details. We introduce MixTok, a hybrid visual tokenizer that combines a compact 1D global representation with sparse 2D tokens for local refinement. A content-adaptive router ranks candidate locations by refinement priority and determines the number of 2D tokens for each image under a target average token budget. At the highest-scoring locations, 2D token features replace the spatial features recovered from the 1D representation, and a shared decoder reconstructs the image from the resulting mixed features. On ImageNet 256 x 256, MixTok achieves highly competitive reconstruction and generation performance, with an rFID of 1.195 and a gFID of 1.95 at respective average budgets of approximately 128 and 131 content tokens per image.

Chenge Li, Yefei He, Xinyu Liu +3

Zhejiang University · The Hong Kong University of Science and Technology · University of Adelaide

Visual Tokenization Visual Token Compression Image Generation

Ask PDF

May 11, 2026

Mutual Enhancement Between Global Tokens and Patch Tokens: From Theory to Practice

Accurate and effective discrete image tokenization is crucial for long image sequence processing. However, current methods rigidly compress all content at a fixed rate, ignoring the variable information density of images and leading to either redundancy or information loss. Inspired by information entropy, we propose TaTok, a Theoretically grounded adaptive image Tokenization framework. We rigorously identify two key drawbacks in existing methods: information insufficiency when reconstructing images with patch tokens alone, and information redundancy among patch tokens. To address these, we introduce global tokens that model mutual information across patch tokens, and a Dynamic Token Filtering (DTF) algorithm based on cumulative conditional entropy to eliminate redundancy. Experiments confirm TaTok's state-of-the-art performance, delivering a 1.3x gFID improvement and 8.7x inference speedup. By allocating tokens according to information richness, TaTok enables more compressed yet accurate image tokenization, offering valuable insights for future research.

Xiusheng Huang, Xin Jiang, Jun Zhao +2

The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Artificial Intelligence, University of Chinese Academy of Sciences · Beijing Academy of Artificial Intelligence

Learned Image Compression Image Tokenization Visual Token Pruning Visual Token Compression

Ask PDF