cs.CVNov 17, 2026

Layer-Matched Sparse Attention for Low-Redundancy 3D Reconstruction

Authors: Sammam Mahdi, Fariha Binta Salim, K. M. Jubair Sami, Rakin Bin Rabbani

Organizations: BRAC University

Abstract

VGGT reconstructs cameras and geometry from unposed images in one pass, but its global attention grows quadratically with the number of views. Training-free accelerators thin this attention and work well on redundant video, but lose pose accuracy when views share little geometry. This paper traces these losses to two kinds of cross-view information that a uniform sparse approximation can fail to preserve together: diffuse layers aggregate distributed context, whereas matching layers depend on query-specific interactions that carry little attention mass. A per-layer statistic measures the log-mass missed by a mean token. Its row-free proxy is the logit variance, and it selects the estimator for each layer. Diffuse layers use shared, importance-sampled hub keys, with log-weights that make the softmax numerator and denominator unbiased. Matching layers stay exact: every aggressive thinning tested cost accuracy. At equal cost, the selected layers outperform the four shifted windows tested. On 13 unseen low-redundancy scenes with 32 views, LMSA stays within 1.4 AUC@5 of dense attention, compared with 10.3 for AVGGT and 4.5 for SAF3R; its difference from contemporaneous GPA-VGGT is not significant, and only its gains over AVGGT pass both scene-level tests. With 100 views, it stays within 1.9 AUC@5 while its aggregator runs 2.0×2.0\times faster. Used post hoc to choose the layers GPA-VGGT protects, the statistic improves it by 1.2 AUC@5 for 5% more FLOPs. Applied untuned to π3π^3, it beats AVGGT but remains further from dense accuracy.

Figures & tables

Appendix figures & tables32 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. VASC: Value-Aware Sparse Attention with Cross-Layer Memory for Efficient 3D Reconstruction

    Oct 1, 2026Junyi Wu, Fanqing Kong, Leyang Chen +23D ViTs3D Reconstruction

  2. RegimeVGGT: Layer-Wise Spatially Preserving Redundancy Removal for Visual Geometry Grounded Transformer

    Jun 16, 2026Jinhao You, Shuo Lyu, Zhuohang Lyu +53D ViTsMulti-View 3D Reconstruction

  3. VGG-T3^3: Offline Feed-Forward 3D Reconstruction at Scale

    Feb 26, 2026Sven Elflein, Ruilong Li, Sérgio Agostinho +43D ReconstructionFeed-Forward 3D Reconstruction