TACT-KV: Tri-Axis Cosine Transform for Compressing Volumetric KV Caches in Medical VLMs
Organizations: Vanderbilt University · NVIDIA · Vanderbilt University Medical Center
Abstract
Vision-language models (VLMs) applied to 3D medical scans face a structural memory wall. A single CT or MRI volume produces tens of thousands of vision tokens, and the resulting key-value (KV) cache scales linearly with both batch and context. Existing KV compression methods operate on the cache's feature or token statistics, or apply 1D and 2D transforms, leaving the three-axis spatial structure in the cache unexploited. We propose TACT-KV (Tri-Axis Cosine Transform), a calibration-free KV compression method that exploits strong spatial correlations in visual KV states along all three anatomical axes of 3D scans. This correlation concentrates the KV cache energy under a frequency-domain transform. TACT-KV applies a separable 3D discrete cosine transform to the KV cache, allocates a per-layer bit budget across frequency bins via reverse water-filling (more bits to high-energy components), and quantizes each bin with a fixed scalar codebook. We derive a close-form relation: the distortion reduction from adaptive bit allocation is dominated by the spectral concentration of the transformed cache. Across six medical VLMs and two 3D CT benchmarks, TACT-KV achieves the lowest 1-bit prediction-flip rate on eleven of twelve model and dataset pairs in our main comparison. The same advantage reproduces on four general-purpose VLMs evaluated on video tasks, indicating that the benefit of exploiting three-axis structure generalizes beyond medical imaging. At equal high-bandwidth memory (HBM), TACT-KV scales the decoding batch size on a single GPU by up to .
Figures & tables
| RSNA-ICH ( ) | CT-RATE ( ) | |||||||
| Method | 1-bit | 2-bit | 3-bit | 4-bit | 1-bit | 2-bit | 3-bit | 4-bit |
| MedGemma 1.5 4B | ||||||||
| TACT-KV (ours) | 3.78 0.28 | 1.91 0.20 | 1.29 0.17 | 1.09 0.15 | 4.69 0.18 | 2.33 0.13 | 1.89 0.11 | 1.54 0.10 |
| TurboQuant | 6.27 0.35 | 4.47 0.30 | 2.16 0.21 | 1.41 0.17 | 9.14 0.24 | 4.61 0.18 | 2.46 0.13 | 1.87 0.11 |
| KIVI | 10.74 0.45 | 1.96 0.20 | 1.12 0.15 | 1.06 0.15 | 23.22 0.35 | 4.66 0.18 | 1.59 0.10 | 1.51 0.10 |
| KVQuant | 9.19 0.42 | 2.31 0.22 | 1.26 0.16 | 1.02 0.15 | 9.46 0.25 | 3.49 0.15 | 1.66 0.11 | 1.51 0.10 |
| RSNA-ICH | CT-RATE | |||||||
| VLM | Closure | Closure | ||||||
| MedGemma 1.5 4B | 0.279 | 4.9% | 0.171 | 28.0% | ||||
| MedGemma 4B | 0.327 | 40.6% | 0.200 | 49.8% | ||||
| MedGemma 27B | 0.265 | 28.6% | 0.180 | 20.6% | ||||
| Hulu-Med 7B | 0.206 | 0.132 | ||||||
| Hulu-Med 4B | 0.253 | 0.162 | ||||||
| MedGemma 1.5 4B | Hulu-Med 7B | |||||||
| Quantizer | 1-bit | 2-bit | 3-bit | 4-bit | 1-bit | 2-bit | 3-bit | 4-bit |
| Gain-shape SQ | 3.78 | 1.91 | 1.29 | 1.09 | 6.08 | 3.51 | 2.33 | 1.86 |
| Uniform sym | 24.04 | 15.56 | 8.45 | 1.41 | 36.25 | 25.88 | 4.94 | 2.12 |
| NormalFloat Dettmers et al. (2023) | 23.26 | 12.40 | 5.34 | 1.29 | 37.05 | 24.41 | 11.74 | 6.60 |
| MedGemma | Hulu-Med | |
| Configuration | 1.5 4B | 7B |
| 3-axis DCT + water-filling (ours) | 3.78 | 6.08 |
| 2-axis DCT + water-filling | 5.00 | 6.21 |
| 1-axis DCT + water-filling | 6.00 | 9.65 |
| w/o DCT + water-filling | 6.61 | 42.20 |
| 3-axis DCT + uniform allocation | 9.97 | 25.76 |
| Method | 1-bit | 2-bit | 3-bit | 4-bit | 1-bit | 2-bit | 3-bit | 4-bit |
| Qwen2.5-VL-7B (bf16: 1.85 ) | Qwen3-VL-8B (bf16: 1.69 ) | |||||||
| TACT-KV (ours) | 57.15 1.87 | 59.52 1.85 | 60.00 1.85 | 59.93 1.85 | 67.56 1.77 | 71.00 1.71 | 71.69 1.70 | 71.93 1.69 |
| TurboQuant | 23.04 1.59 | 25.04 1.63 | 21.33 1.55 | 44.85 1.88 | 14.33 1.32 | 54.26 1.88 | 70.93 1.71 | 71.67 1.70 |
| KIVI | 12.04 1.23 | 59.70 1.85 | 60.37 1.84 | 59.96 1.85 | 3.44 0.69 | 71.30 1.71 | 71.89 1.70 | 72.22 1.69 |
| VidKV | 19.70 1.50 | 59.33 1.85 | – | – | 26.63 1.67 | 71.67 1.70 | – | – |
| AKVQ-VL | 39.44 1.84 | 59.89 1.85 | – | – | 7.37 0.99 | 71.37 1.71 | – | – |
| Method | Max decoding | Per-request cache (MB) | Decode step (ms) | Throughput (tok/s) |
| Hulu-Med 4B (Weights 9.0 GB, HBM headroom 93.6 GB) | ||||
| bf16 | 32 | 2333 | 165 | 194 |
| TACT-KV | 128 | 309 | 467 | 274 |
| Hulu-Med 7B (Weights 16.1 GB, HBM headroom 86.5 GB) | ||||
| bf16 | 64 | 915 | 136 | 472 |
| TACT-KV | 400 | 132 | 723 | 553 |
| Decode step (ms) | |||
| Model | bf16 | TACT-KV | |
| Hulu-Med 4B | 32 | 165 | 114 |
| Hulu-Med 7B | 64 | 136 | 119 |
| Hulu-Med 32B | 8 | 121 | 110 |
Appendix figures & tables22 assets
Supplementary material from the paper’s appendix.
Appendix
| RSNA-ICH | CT-RATE | |||||||
| VLM | ||||||||
| MedGemma 1.5 4B | 0.279 | 0.389 | 0.458 | 0.412 | 0.171 | 0.215 | 0.473 | 0.447 |
| MedGemma 4B | 0.327 | 0.448 | 0.534 | 0.468 | 0.200 | 0.249 | 0.534 | 0.480 |
| MedGemma 27B | 0.265 | 0.443 | 0.440 | 0.379 | 0.180 | 0.258 | 0.432 | 0.396 |
| Hulu-Med 7B | 0.206 | 0.416 | 0.354 | 0.273 | 0.132 | 0.279 | 0.345 | 0.271 |
| Hulu-Med 4B | 0.253 | 0.444 | 0.392 | 0.336 | 0.162 | 0.282 | 0.367 | 0.320 |
| Training pool ( ) | Subset ( ) | ||||
| Label | positives | rate (%) | positives | rate (%) | (pp) |
| any | |||||
| epidural | |||||
| intraparenchymal | |||||
| intraventricular | |||||
| subarachnoid | |||||
| Subtype | bf16 | TACT-KV | TQ | KIVI | VidKV | AKVQ-VL |
| Pred-flip rate (%) vs bf16 | ||||||
| any | — | 2.33 0.54 | 3.73 0.68 | 4.27 0.72 | 4.07 0.71 | 14.61 1.26 |
| epidural | — | 0.83 0.32 | 2.03 0.50 | 5.97 0.85 | 1.83 0.48 | 31.92 1.67 |
| intraparenchymal | — | 5.60 0.82 | 7.77 0.96 | 10.57 1.10 | 10.11 1.08 | 27.42 1.60 |
| intraventricular | — | 7.77 0.96 | 10.27 1.09 | 19.94 1.43 | 13.88 1.24 | 37.56 1.73 |
| subarachnoid | — | 1.13 0.38 | 4.80 0.77 | 7.90 0.97 | 4.77 0.76 | 33.69 1.69 |
| Subtype | bf16 | TACT-KV | TQ | KIVI | VidKV | AKVQ-VL |
| Pred-flip rate (%) vs bf16 | ||||||
| any | — | 1.70 0.46 | 4.60 0.75 | 1.57 0.44 | 17.14 1.35 | 0.30 0.20 |
| epidural | — | 9.87 1.07 | 13.80 1.23 | 11.77 1.15 | 31.19 1.66 | 12.34 1.18 |
| intraparenchymal | — | 1.20 0.39 | 4.67 0.76 | 0.30 0.20 | 14.54 1.26 | 0.13 0.13 |
| intraventricular | — | 0.00 0.07 | 0.00 0.07 | 0.03 0.06 | 1.53 0.44 | 0.00 0.07 |
| subarachnoid | — | 1.27 0.40 | 3.27 0.64 | 0.77 0.31 | 17.41 1.36 | 0.37 0.22 |
| Subtype | bf16 | TACT-KV | TQ | KIVI | VidKV | AKVQ-VL |
| Pred-flip rate (%) vs bf16 | ||||||
| any | — | 0.00 0.07 | 0.00 0.07 | 0.53 0.26 | 0.00 0.07 | 6.94 0.91 |
| epidural | — | 0.53 0.26 | 9.04 1.03 | 96.00 0.70 | 1.77 0.47 | 70.01 1.64 |
| intraparenchymal | — | 0.00 0.07 | 0.03 0.06 | 0.13 0.13 | 0.00 0.07 | 8.24 0.98 |
| intraventricular | — | 0.20 0.16 | 8.17 0.98 | 0.33 0.21 | 9.57 1.05 | 2.87 0.60 |
| subarachnoid | — | 0.00 0.07 | 0.00 0.07 | 0.00 0.07 | 0.00 0.07 | 0.23 0.17 |
| Subtype | bf16 | TACT-KV | TQ | KIVI | VidKV | AKVQ-VL |
| Pred-flip rate (%) vs bf16 | ||||||
| any | — | 16.07 1.31 | 77.93 1.48 | 77.99 1.48 | 27.79 1.60 | 75.45 1.54 |
| epidural | — | 19.47 1.42 | 27.74 1.60 | 28.41 1.61 | 28.62 1.62 | 28.39 1.61 |
| intraparenchymal | — | 2.07 0.51 | 94.83 0.79 | 95.40 0.75 | 34.92 1.71 | 93.70 0.87 |
| intraventricular | — | 0.00 0.07 | 96.93 0.62 | 95.60 0.73 | 0.17 0.15 | 94.76 0.80 |
| subarachnoid | — | 0.03 0.06 | 96.77 0.63 | 96.97 0.61 | 44.26 1.78 | 95.00 0.78 |
| Subtype | bf16 | TACT-KV | TQ | KIVI | VidKV | AKVQ-VL |
| Pred-flip rate (%) vs bf16 | ||||||
| any | — | 6.14 0.86 | 34.11 1.70 | 27.04 1.59 | 22.18 1.49 | 35.76 1.72 |
| epidural | — | 0.10 0.11 | 25.61 1.56 | 6.64 0.89 | 2.43 0.55 | 13.68 1.23 |
| intraparenchymal | — | 4.83 0.77 | 31.48 1.66 | 17.44 1.36 | 12.74 1.19 | 29.59 1.63 |
| intraventricular | — | 14.40 1.26 | 66.72 1.69 | 71.09 1.62 | 67.75 1.67 | 63.18 1.73 |
| subarachnoid | — | 9.67 1.06 | 33.81 1.69 | 21.37 1.47 | 18.85 1.40 | 31.32 1.66 |
| Subtype | bf16 | TACT-KV | TQ | KIVI | VidKV | AKVQ-VL |
| Pred-flip rate (%) vs bf16 | ||||||
| any | — | 6.27 0.87 | 12.57 1.19 | 31.41 1.66 | 26.98 1.59 | 42.56 1.77 |
| epidural | — | 8.57 1.00 | 12.77 1.19 | 15.67 1.30 | 15.04 1.28 | 34.59 1.70 |
| intraparenchymal | — | 3.93 0.70 | 9.60 1.05 | 21.94 1.48 | 14.34 1.25 | 43.46 1.77 |
| intraventricular | — | 18.54 1.39 | 34.08 1.70 | 69.49 1.65 | 59.84 1.75 | 27.75 1.60 |
| subarachnoid | — | 4.80 0.77 | 58.75 1.76 | 92.66 0.93 | 65.08 1.71 | 29.72 1.64 |
| Abnormality | bf16 | TACT-KV | TQ | KIVI | VidKV | AKVQ-VL |
| Pred-flip rate (%) vs bf16 | ||||||
| Medical material | — | 0.00 0.07 | 0.00 0.07 | 0.59 0.27 | 0.00 0.07 | 8.00 0.96 |
| Arterial wall calcification | — | 0.00 0.07 | 6.55 0.88 | 26.82 1.58 | 0.07 0.09 | 17.14 1.34 |
| Cardiomegaly | — | 3.85 0.68 | 2.96 0.60 | 39.06 1.73 | 5.56 0.81 | 21.19 1.45 |
| Pericardial effusion | — | 6.84 0.90 | 16.16 1.31 | 53.90 1.77 | 6.84 0.90 | 16.85 1.33 |
| Coronary artery wall calcification | — | 19.97 1.42 | 34.02 1.68 | 48.37 1.78 | 20.11 1.43 | 36.72 1.71 |
| Abnormality | bf16 | TACT-KV | TQ | KIVI | VidKV | AKVQ-VL |
| Pred-flip rate (%) vs bf16 | ||||||
| Medical material | — | 0.00 0.07 | 0.00 0.07 | 0.56 0.27 | 0.00 0.07 | 1.45 0.43 |
| Arterial wall calcification | — | 0.03 0.06 | 0.07 0.09 | 0.69 0.29 | 0.26 0.18 | 17.34 1.35 |
| Cardiomegaly | — | 0.13 0.13 | 0.16 0.14 | 6.75 0.89 | 1.09 0.37 | 12.77 1.19 |
| Pericardial effusion | — | 0.03 0.06 | 0.00 0.07 | 0.23 0.17 | 0.20 0.16 | 15.60 1.29 |
| Coronary artery wall calcification | — | 0.43 0.23 | 0.39 0.22 | 13.16 1.20 | 0.76 0.31 | 25.77 1.56 |
| Abnormality | bf16 | TACT-KV | TQ | KIVI | VidKV | AKVQ-VL |
| Pred-flip rate (%) vs bf16 | ||||||
| Medical material | — | 0.03 0.06 | 0.07 0.09 | 3.39 0.64 | 0.33 0.20 | 3.59 0.66 |
| Arterial wall calcification | — | 0.43 0.23 | 12.47 1.17 | 19.45 1.41 | 18.33 1.38 | 16.42 1.32 |
| Cardiomegaly | — | 0.00 0.07 | 0.53 0.26 | 8.16 0.97 | 6.91 0.90 | 0.16 0.14 |
| Pericardial effusion | — | 10.86 1.11 | 22.28 1.48 | 24.25 1.52 | 24.25 1.52 | 21.36 1.46 |
| Coronary artery wall calcification | — | 3.13 0.62 | 5.20 0.79 | 5.20 0.79 | 5.20 0.79 | 5.20 0.79 |
| Abnormality | bf16 | TACT-KV | TQ | KIVI | VidKV | AKVQ-VL |
| Pred-flip rate (%) vs bf16 | ||||||
| Medical material | — | 0.00 0.07 | 18.46 1.38 | 2.47 0.55 | 0.00 0.07 | 4.21 0.71 |
| Arterial wall calcification | — | 23.69 1.51 | 43.24 1.76 | 44.16 1.77 | 50.61 1.78 | 43.30 1.76 |
| Cardiomegaly | — | 8.06 0.97 | 69.23 1.64 | 78.87 1.45 | 67.49 1.67 | 77.76 1.48 |
| Pericardial effusion | — | 0.00 0.07 | 92.76 0.92 | 98.55 0.43 | 2.83 0.59 | 90.85 1.03 |
| Coronary artery wall calcification | — | 10.99 1.11 | 14.35 1.25 | 14.94 1.27 | 72.89 1.58 | 14.45 1.25 |
| Abnormality | bf16 | TACT-KV | TQ | KIVI | VidKV | AKVQ-VL |
| Pred-flip rate (%) vs bf16 | ||||||
| Medical material | — | 18.36 1.38 | 38.01 1.73 | 38.80 1.73 | 37.97 1.73 | 38.37 1.73 |
| Arterial wall calcification | — | 9.74 1.05 | 60.15 1.74 | 90.62 1.04 | 12.90 1.19 | 90.06 1.06 |
| Cardiomegaly | — | 0.99 0.35 | 85.36 1.26 | 84.40 1.29 | 21.85 1.47 | 88.25 1.14 |
| Pericardial effusion | — | 1.22 0.39 | 76.74 1.50 | 89.63 1.08 | 8.49 0.99 | 95.33 0.75 |
| Coronary artery wall calcification | — | 5.20 0.79 | 55.48 1.77 | 91.94 0.97 | 26.59 1.57 | 84.93 1.27 |
| Abnormality | bf16 | TACT-KV | TQ | KIVI | VidKV | AKVQ-VL |
| Pred-flip rate (%) vs bf16 | ||||||
| Medical material | — | 32.97 1.67 | 36.69 1.71 | 49.16 1.78 | 32.08 1.66 | 49.00 1.78 |
| Arterial wall calcification | — | 0.36 0.21 | 23.86 1.52 | 98.26 0.46 | 27.57 1.59 | 79.66 1.43 |
| Cardiomegaly | — | 13.92 1.23 | 29.94 1.63 | 27.74 1.59 | 49.75 1.78 | 28.00 1.60 |
| Pericardial effusion | — | 0.00 0.07 | 9.05 1.02 | 97.89 0.51 | 15.53 1.29 | 43.99 1.76 |
| Coronary artery wall calcification | — | 11.94 1.15 | 24.88 1.54 | 71.21 1.61 | 20.63 1.44 | 62.78 1.72 |
| Task type | bf16 | TACT-KV | TQ | KIVI | VidKV | AKVQ-VL |
| Temporal Perception | 65.45 12.20 | 60.00 12.53 | 12.73 8.86 | 16.36 9.70 | 25.45 11.24 | 36.36 12.32 |
| Spatial Perception | 64.81 12.35 | 62.96 12.47 | 24.07 11.16 | 12.96 9.00 | 29.63 11.84 | 44.44 12.81 |
| Attribute Perception | 74.32 5.71 | 70.27 5.98 | 24.77 5.65 | 12.61 4.38 | 18.47 5.08 | 45.50 6.50 |
| Action Recognition | 56.87 5.46 | 54.63 5.48 | 19.49 4.38 | 15.34 3.99 | 20.45 4.46 | 37.70 5.34 |
| Object Recognition | 64.41 4.96 | 60.73 5.06 | 24.58 4.47 | 15.54 3.77 | 19.21 4.09 | 38.98 5.05 |
| OCR Problems | 66.19 7.77 | 60.43 8.03 | 22.30 6.87 | 10.79 5.19 | 22.30 6.87 | 39.57 8.02 |
| Task type | bf16 | TACT-KV | TQ | KIVI | VidKV | AKVQ-VL |
| Temporal Perception | 78.18 10.71 | 74.55 11.24 | 12.73 8.86 | 3.64 5.66 | 23.64 10.99 | 9.09 7.81 |
| Spatial Perception | 77.78 10.87 | 74.07 11.40 | 11.11 8.50 | 5.56 6.60 | 46.30 12.85 | 3.70 5.76 |
| Attribute Perception | 83.78 4.84 | 78.38 5.39 | 18.92 5.13 | 3.15 2.41 | 33.33 6.16 | 5.86 3.15 |
| Action Recognition | 71.25 4.99 | 67.09 5.18 | 16.29 4.08 | 2.56 1.83 | 30.03 5.05 | 5.43 2.55 |
| Object Recognition | 76.27 4.42 | 71.75 4.67 | 17.80 3.98 | 2.26 1.63 | 34.46 4.93 | 10.45 3.20 |
| OCR Problems | 77.70 6.86 | 72.66 7.34 | 12.23 5.46 | 0.72 1.92 | 34.53 7.81 | 6.47 4.21 |
| Task type | bf16 | TACT-KV | TQ | KIVI | VidKV | AKVQ-VL |
| Temporal Perception | 61.82 12.44 | 54.55 12.72 | 25.45 11.24 | 9.09 7.81 | 32.73 12.04 | 18.18 10.07 |
| Spatial Perception | 61.11 12.58 | 51.85 12.88 | 53.70 12.85 | 11.11 8.50 | 55.56 12.81 | 29.63 11.84 |
| Attribute Perception | 75.68 5.61 | 69.82 6.00 | 50.90 6.52 | 9.46 3.88 | 39.64 6.39 | 22.07 5.43 |
| Action Recognition | 54.95 5.48 | 54.31 5.48 | 37.38 5.33 | 7.99 3.02 | 37.06 5.32 | 19.17 4.35 |
| Object Recognition | 66.95 4.88 | 58.76 5.10 | 47.46 5.17 | 10.17 3.16 | 42.66 5.12 | 20.62 4.21 |
| OCR Problems | 60.43 8.03 | 54.68 8.16 | 37.41 7.94 | 5.76 4.00 | 42.45 8.11 | 18.71 6.45 |
| Task type | bf16 | TACT-KV | TQ | KIVI | VidKV | AKVQ-VL |
| Temporal Perception | 65.45 12.20 | 65.45 12.20 | 38.18 12.44 | 9.09 7.81 | 40.00 12.53 | 12.73 8.86 |
| Spatial Perception | 64.81 12.35 | 62.96 12.47 | 51.85 12.88 | 9.26 7.95 | 48.15 12.88 | 22.22 10.87 |
| Attribute Perception | 76.58 5.55 | 72.07 5.87 | 44.59 6.48 | 9.91 3.96 | 40.99 6.42 | 13.06 4.44 |
| Action Recognition | 67.09 5.18 | 64.54 5.27 | 41.53 5.43 | 5.75 2.62 | 39.62 5.39 | 13.42 3.78 |
| Object Recognition | 71.19 4.70 | 66.95 4.88 | 40.68 5.09 | 8.47 2.92 | 39.27 5.06 | 14.69 3.69 |
| OCR Problems | 73.38 7.27 | 65.47 7.81 | 42.45 8.11 | 5.76 4.00 | 38.13 7.97 | 13.67 5.71 |
| Method | RSNA-ICH | CT-RATE |
| MedGemma 1.5 4B | ||
| TACT-KV (ours) | 3.78 0.28 | 4.69 0.18 |
| VL-Cache | 2.77 0.24 | 4.21 0.17 |
| FlashCache | 8.27 0.40 | 9.35 0.24 |
| MedGemma 4B | ||
| TACT-KV (ours) | 2.37 0.22 | 2.95 0.14 |
| RSNA-ICH | CT-RATE | |||||
| Model | 2-bit | 4-bit | 2-bit | 4-bit | ||
| MedGemma 1.5 4B | 3.58 | 2.97 | 2.65 | 5.85 | 8.76 | 7.02 |
| MedGemma 4B | 3.06 | 2.84 | 2.45 | 5.00 | 7.05 | 5.80 |
| MedGemma 27B | 3.77 | 2.82 | 2.61 | 5.56 | 6.08 | 5.42 |
| Hulu-Med 7B | 4.85 | 4.01 | 3.61 | 7.58 | 10.35 | 8.45 |
| Hulu-Med 4B | 3.95 | 3.23 | 3.04 | 6.17 | 9.01 | 7.99 |
| Quantizer | ||||
| gain-shape SQ (TACT-KV) | ||||
| Uniform sym | ||||
| NormalFloat |
| range | # coordinates (%) | Uniform MAE | gain-shape SQ MAE | SQ advantage |
| (the bulk) | ( ) | |||
| ( ) | ||||
| ( ) | ||||
| ( ) | ||||
| ( ) | ||||
| ( ) | SQ worse |
KV-Cache Compression
Key-value cache compression reduces the memory and bandwidth needed to retain attention states during long-context inference.
See all 290 papers