Rotation Is Token-Blind: A Structural Criterion for Post-Training Quantization of Feed-Forward Geometry Transformers
Organizations: School of Future Science and Engineering, Soochow University, Suzhou, China · D-Robotics, Beijing, China · School of Mathematical and Computing Sciences, Heriot-Watt University Malaysia, Putrajaya, Malaysia · Fraunhofer Institute for Applied Information Technology, Sankt Augustin, Germany
Abstract
Feed-forward 3D models use Transformers to predict camera poses, depth, and structure in one pass, avoiding iterative multi-view optimization. Long sequences and deep attention incur high compute and memory costs, motivating Post-Training Quantization (PTQ) for deployment. Existing PTQ methods mainly suppress channel-wise outliers by smoothing or orthogonal rotation. Geometry Transformers introduce a different challenge: heterogeneous token types share one activation quantization grid. A few special tokens may exert high leverage over global outputs and be updated by only a subset of blocks. Token-blind transforms, which apply the same transformation to every token, reduce overall quantization error but cannot remove differentiated degradation across token types. We propose QMapAnything, a structure-aware PTQ framework. We cast floating-point-preserving rotation and smoothing as a token-blind reparameterization group, and prove that they yield common-mode gains while leaving relative token SNR unchanged. We introduce a statistic that combines a special token's quantization disadvantage and output leverage to predict whether differentiated bias correction is needed beyond rotation. Experiments on geometric benchmarks show that W4A8 closely matches full-precision performance. Under W8A8, peak memory falls by -- versus FP32, with up to speedup on real hardware. These results provide a principled and efficient approach to deploying feed-forward 3D reconstruction models in resource-constrained edges.
Figures & tables
| Method | W/A | frames=2 | frames=8 | frames=16 |
|---|---|---|---|---|
| AUC@30 ∘ | AUC@30 ∘ | AUC@30 ∘ | ||
| Full Prec. | 16/16 | 98.9 | 99.2 | 98.8 |
| RTN | 4/8 | 91.8 | 89.2 | 88.1 |
| SmoothQuant | 4/8 | 96.9 | 94.9 | 94.1 |
| QuaRot | 4/8 | 98.5 | 98.5 | 98.1 |
| QMapAnything | 4/8 | 98.6 | 98.6 | 98.1 |
| Method | W/A | Scale | Points | Pose | Depth | Rays | |||
|---|---|---|---|---|---|---|---|---|---|
| rel | rel | ATE | AUC | rel | err | ||||
| Full Prec. | 16/16 | 0.05 | 0.06 | 60.3 | 0.01 | 98.9 | 0.06 | 57.3 | 0.19 |
| RTN | 4/8 | 0.10 | 0.09 | 43.1 | 0.02 | 91.8 | 0.08 | 37.4 | 0.57 |
| SmoothQuant | 4/8 | 0.06 | 0.07 | 52.5 | 0.02 | 96.9 | 0.06 | 48.1 | 0.34 |
| QuaRot | 4/8 | 0.05 | 0.06 | 59.1 | 0.01 | 98.5 | 0.06 | 55.2 | 0.22 |
| QMapAnything | 4/8 | 0.05 | 0.06 | 59.2 | 0.01 | 98.6 | 0.06 | 56.0 | 0.19 |
| R | ETH3D | SN++v2 | TAV2 | ||
|---|---|---|---|---|---|
| AUC@30 | AUC@30 | AUC@30 | |||
| 77.0 | 74.3 | 64.0 | |||
| ✓ | 83.7 | 86.9 | 86.1 | ||
| ✓ | ✓ | 87.3 | 88.5 | 89.1 | |
| ✓ | ✓ | 88.2 | 90.4 | 89.4 | |
| ✓ | ✓ | ✓ | 91.4 | 93.7 | 93.2 |
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
| Rotation | (A) Orth. | (B) Flat | (C) Rand. |
|---|---|---|---|
| Identity | |||
| Discrete cosine transform (DCT) | ✓ | ||
| Fixed Hadamard ( ) | ✓ | ✓ | |
| Haar random orthogonal | ✓ | ✓ | |
| Randomized Hadamard ( ) | ✓ | ✓ | ✓ |
Post-Training Quantization
Post-Training Quantization compresses trained neural networks into low-bit weights, activations, or caches to reduce memory use and inference cost.
See all 395 papers