VoluCompose: Constructing Compact Volumetric Visual Representations for Medical Image Understanding
Authors: Zanting Ye, Yunrui Lin, Shengyuan Liu, Jing Hao, Chen Hu, Yefeng Zheng, Lijun Lu
Organizations: School of Biomedical Engineering and Guangdong Provincial Key Laboratory of Medical Image Processing, Southern Medical University, Guangzhou 510515, China · Medical Artificial Intelligence Laboratory, Westlake University, Hangzhou 310030, China · Department of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong SAR, China · Faculty of Dentistry, The University of Hong Kong, Hong Kong SAR, China
Medical vision--language models can interpret volumetric images, but processing full volumes is costly because hundreds of slices produce long, redundant visual-token sequences. Existing compression methods often couple content selection with budget determination. We observe that neighboring slices share visual features and that attention concentration varies across visual encoders. We propose VoluCompose, a training-free, model-agnostic framework that separates what visual content to retain from how many tokens to allocate. Its hyperbolic token composition module groups contiguous slices into ordered support units and uses the Lorentz distance to account for feature magnitude and direction during token allocation, anchor selection, and refinement. Its capacity control module uses visual-encoder attention entropy to allocate more tokens when attention is more dispersed. We evaluate VoluCompose on two tasks across three datasets and four backbones. Across nine calibrated settings, VoluCompose attains the highest mean quality--retention hypervolume among the compared methods. It achieves a mean relative quality of 121.68% against the uncompressed references while retaining 22.78% of visual tokens on average. Compression remains robust even at a 10% token budget. Across settings, the lowest ROUGE-L ratio to the uncompressed reference is 96.4% for VoluCompose. The corresponding minima for the compression baselines range from 15.0% to 70.5%. Code is available at https://github.com/yezanting/VoluCompose.
Figures & tables
Figure 1: Cross-slice feature recurrence, attention heterogeneity, and performance under compact token budgets. (a) Adjacent-minus-nonadjacent cosine similarity of slice features; bars show means with 95% bootstrap intervals over 50 studies per dataset. (b) Setting-level ROUGE-L of VoluCompose relative to the uncompressed reference (100%) under fixed nominal budgets (10%, 20%, and 30%). (c)–(f) Ranked attention density averaged over 150 profiles per backbone; annotations give the attention mass in the top 5% of tokens.
Figure 2: Overview of VoluCompose. A frozen visual interface converts an ordered volume into visual tokens. The hyperbolic token composition module groups contiguous slices into ordered support units, allocates tokens across units, selects distinctive anchors with the Lorentz distance, and refines them with related context while preserving source order. The attention-calibrated capacity control module determines the total budget B from a prescribed retention fraction or normalized attention entropy. The compact token sequence is supplied to the language decoder for downstream tasks.
3D-RAD [ 15 ]
M3D [ 2 ]
AMOS-MM [ 17 ]
Overall
Worst case (%)
Method
ROUGE-L
Close-Acc
ROUGE-L
Close-Acc
ROUGE-L
METEOR
ROUGE-L
Close-Acc
ROUGE-L
Close-Acc
Uncompressed
18.09
55.62
25.34
55.83
19.01
18.91
20.81
55.73
–
–
HoliTom [ 33 ]
21.50
52.57
29.31
57.96
17.57
13.28
22.79
55.27
37.7
84.0
VisionZip [ 36 ]
21.43
53.93
28.99
59.04
15.91
11.83
22.11
56.48
15.0
81.3
UniComp [ 40 ]
21.15
51.31
25.39
55.54
17.63
14.36
21.39
53.42
70.5
78.8
VoluCompose
21.67
56.65
28.89
57.78
23.57
17.91
24.71
57.22
96.4
94.2
Table 1: Performance at 10% nominal retention. Under the fixed protocol, dataset performance is averaged over four backbones, while overall ROUGE-L and closed-question accuracy (Close-Acc) are averaged across 12 and eight settings, respectively. Worst case is the lowest setting-level score as a percentage of the corresponding uncompressed score. Bold marks the best compression result in each column.
3D-RAD [ 15 ]
M3D [ 2 ]
AMOS-MM [ 17 ]
Method
ROUGE-L
Close-Acc
ROUGE-L
Close-Acc
ROUGE-L
METEOR
ROUGE-1
BLEU-1
BLEU-4
Hulu-7B
Uncompressed
22.92
78.77
34.05
75.32
28.76
31.73
39.52
37.32
13.38
HoliTom [ 33 ]
31.86
77.94
46.15
76.46
27.21
19.52
28.56
23.16
5.89
VisionZip [ 36 ]
31.47
78.84
45.19
75.65
26.60
18.48
27.99
22.53
4.95
UniComp [ 40 ]
30.36
76.23
41.46
74.14
20.28
13.69
21.38
14.43
2.44
Table 2: Per-backbone performance at 10% nominal retention. Visual question answering (VQA) is evaluated on 3D-RAD and M3D, and report generation on AMOS-MM. Bold marks the highest score among compression methods in each column and backbone.
3D-RAD [ 15 ]
M3D [ 2 ]
AMOS-MM [ 17 ]
Overall
Method
Retention rate (%)
ROUGE-L
Close-Acc
Retention rate (%)
ROUGE-L
Close-Acc
Retention rate (%)
ROUGE-L
METEOR
Q (%)
HV
Hulu-7B
Uncompressed
100.00
22.92
78.77
100.00
34.05
75.32
100.00
28.76
31.73
100.00
0.000
Hulu-Med L1 [ 18 ]
45.06
23.49
78.08
66.03
47.17
77.79
16.35
24.83
26.21
100.65
0.547
HiPrune [ 23 ]
22.30
25.56
79.06
22.30
46.20
77.62
22.30
27.96
31.09
106.91
0.831
MedPruner [ 25 ]
51.88
25.98
79.28
52.10
47.26
77.45
54.20
28.86
31.14
108.50
0.514
Table 3: Performance and realized token retention at calibrated operating points. Q and HV use the corresponding uncompressed scores. Overall columns average the settings in each block. Bold marks the highest Q and HV among compressors.
Figure 3: Performance under fixed and calibrated token budgets. (a) Dataset means over four backbones. (b) Mean relative quality Q across 12 fixed-budget settings per budget. Dashed lines in (a) and (b) mark uncompressed references. (c) Differences in ROUGE-L and realized token retention (VoluCompose minus MedPruner [ 25 ] ) across nine calibrated settings. Upper-left points have higher ROUGE-L and lower retention. pp: percentage points.
Figure 4: Ordered support construction and anchor refinement. (a) A 24-slice Hulu-7B [ 18 ] input forms 11 ordered support units; the curve shows per-unit token allocation. (b), (c) Support units containing slices 6 and 22 across four backbones. The Anchors row marks selected anchors in red. The Pair 1–3 rows show anchor–neighborhood examples in matching colors. Related patches can be spatially separated; refinement preserves the selected token count.
Figure 5: Visual–text correlation maps for all and retained tokens. Each row shows two examples from one backbone. Paired maps use the same full text and color scale; gray marks excluded positions. Percentages indicate whole-volume token retention.
Full
Anchor relation
Allocation
Refinement
Metric
Score
Cosine
Euclidean
Uniform
Removed
Hulu-7B
ROUGE-L
37.41
+15.42
+17.39
+1.95
+2.21
METEOR
28.35
+13.02
+13.21
+3.72
+2.15
MedGemma-1.5-4B
ROUGE-L
17.99
−1.67
+1.04
+2.34
+2.15
Table 4: Component ablations on AMOS-MM at 10% nominal retention with matched per-study token counts. Full gives the absolute scores of VoluCompose; the other columns give Full minus each control, which uses a cosine or Euclidean anchor relation, uniform allocation, or no refinement. Positive values favor VoluCompose.
Visual token compression reduces image and video representations to lower inference, memory, storage, or communication costs while retaining task-relevant information.
Slice-based MLLMs leverage mature 2D encoders by representing 3D volumes as sequences of 2D slices. However, this slice-wise formulation produces thousands of visual tokens that burden the LLM backbone, many of which capture overlapping visual evidence across adjacent slices. To understand how effectively a growing visual token budget improves performance, we perform scaling analyses on two 3D medical VQA benchmarks and find diminishing returns: cost keeps rising while accuracy saturates, and improving in-plane resolution is more effective than adding slices at comparable budgets. The budget should therefore be allocated more selectively rather than simply enlarged, yet most token compression methods are designed for 2D images or videos, where redundancy arises from spatial layout or temporal motion rather than from near-duplicate content along the depth axis. We present CARVE, a training-free framework that compresses visual tokens prior to LLM inference and casts token reduction as budget-constrained 2.5D allocation. CARVE partitions the depth axis into coherent windows and allocates tokens non-uniformly according to normalized cross-slice evidence. Under a shared budget, CARVE builds spatial anchors on representative slices and retrieves locally varying evidence from the full volume, then merges remaining eligible tokens into nearby anchors within each window. Removing roughly 80% of the visual tokens on Hulu-Med-7B, CARVE leads all compression baselines on every AMOS-MM report-generation metric, with 6.2 points higher retention of full-token quality than the strongest baseline, and preserves 98.1% of full-token performance across three VQA benchmarks.
Zhenyu Yi, Qiang Hu, Zhenhao Li +3
School of Biomedical Engineering, Shanghai Jiao Tong University · 2Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology
Integrating 3D medical images with vision-language models (VLMs) holds substantial promise for computer-aided diagnosis. However, volumetric images generate prohibitively long visual-token sequences with considerable spatial and inter-slice redundancy. Existing token compression methods typically apply uniform reduction or rely on a single importance signal, increasing the risk of removing regions that are clinically relevant to the query or structurally distinctive. To address this limitation, we propose MedARC, a unified, training-free framework for Adaptive Redundancy Compression of visual tokens in 3D medical VLMs. MedARC estimates token importance by integrating three complementary cues: self-attention from the VLM vision encoder, which reflects the model's intrinsic visual focus; similarity between projected visual tokens and text embeddings, which identifies query-relevant regions; and deviations of local visual foundation model features from the volume-level feature center, which highlight structurally distinctive anatomy. The resulting importance distribution guides a saliency-aware merging strategy that preserves informative tokens while consolidating redundant ones rather than simply discarding them. Experiments on CT-RATE and MR-RATE show that MedARC reduces visual-token overhead and inference time while preserving or improving diagnostic performance. Its multi-cue scoring cost is outweighed by the savings from processing fewer tokens, with greater benefits expected for larger language models.
Yitao Zhu, Mengjun Liu, Yingji Fu +2
Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Hong Kong · School of Integrated Circuits and Electronics, Beijing Institute of Technology, China · 3Mental Health Research Center, The Hong Kong Polytechnic University, Hong Kong +1
Visual tokens are a major source of inference cost in vision-language models, yet simple image downsampling remains a surprisingly strong compression baseline. This raises a complementary question: under a fixed token budget, where should visual fidelity be preserved? We introduce Foveated Compression, which encodes a full-resolution image once and represents it with a mixture of native- and compressed-resolution visual tokens. A behaviorally self-distilled Foveated Merger compresses local visual tokens while preserving compatibility with their native counterparts, and a lightweight Foveated Selector chooses one of nine spatial cells to retain at native resolution using exhaustive budget-matched intervention supervision. At 11.11% visual tokens, uniform Foveated Compression shows no significant paired difference from iso-token downsampling. At 20.99%, the learned selector significantly outperforms random and fixed allocation, but remains below strong whole-image resizing, showing that localized fidelity is not universally preferable. A budget-matched region-choice oracle reaches 82.73 macro accuracy versus 69.61 for the learned selector, revealing substantial headroom within the same spatial action space. Matched probing further shows that signals predicting when compression breaks the answer are substantially more accessible after language-model computation than to the lightweight prefill-free selector. These results expose complementary bottlenecks in region selection and compressed-region fidelity.
Donghyun Han, Jangho Park, Yuseok Bae
ETRI, South Korea · Kyung Hee University, South Korea