cs.CVNov 15, 2026

AGPF: Dynamic Source Selection for Feature Pyramid Fusion

Authors: Xu Yang, Gady Agam

Organizations: Department of Computer Science Illinois Institute of Technology

Abstract

Feature pyramids provide dense predictors with features at several spatial resolutions. However, it remains largely unexplored which pyramid levels each output location should aggregate information from. We investigate this question by allowing each output location to attend to features from every pyramid level and measuring how attention is distributed across scales. Surprisingly, locations on the finest output grid allocate approximately 75% of their attention to features two levels coarser, rather than to the current or adjacent levels. Even on the same output grid, the balance shifts toward coarser features as object size increases. These observations motivate Asymmetric Global Pyramid Fusion (AGPF), a neck that preserves the original output grids while allowing each location to dynamically select its contextual sources across scales. Each location interacts with a multiscale memory whose features are aligned in image coordinates, enabling flexible cross-scale aggregation without altering prediction resolutions. Compared with FPN, AGPF improves all 19 object detection and all seven instance segmentation configurations evaluated on COCO. A single fusion block improves Cascade Mask R-CNN by 1.7 box AP and 2.3 mask AP, and the gains consistently transfer to semantic and panoptic segmentation.

Figures & tables

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. A3-FPN: Asymptotic Content-Aware Pyramid Attention Network for Dense Visual Prediction

    Apr 11, 2026Meng'en Qin, Yu Song, Quanling Zhao +3Dense PredictionMulti-Scale Feature Fusion

  2. Context-Guided Semantic Alignment for Feature Fusion Networks

    Jun 12, 2026Hyungseop Lee, Jiho Lee, Woochul KangMulti-Scale Feature FusionObject Detection

  3. Weave Mamba Fusion: Global Cross-Scale Interaction for Lightweight Face Detection

    Oct 5, 2026Dohun Kim, Jinmyung JungEfficient Object DetectionMulti-Scale Feature Fusion