A3-FPN: Asymptotic Content-Aware Pyramid Attention Network for Dense Visual Prediction

Meng'en Qin; Yu Song; Quanling Zhao; Xiaodong Yang; Yingtao Che; Xiaohui Yang

cs.CVApr 11, 2026

A3-FPN: Asymptotic Content-Aware Pyramid Attention Network for Dense Visual Prediction

Authors: Meng'en Qin, Yu Song, Quanling Zhao, Xiaodong Yang, Yingtao Che, Xiaohui Yang

Organizations: Henan Engineering Research Center for Artificial Intelligence Theory and Algorithms, Henan University, Kaifeng, China · Faculty of Computer Science and Control Engineering, Shenzhen University of Advanced Technology, Shenzhen, China · Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University, Hong Kong, China

Abstract

Learning multi-scale representations is the common strategy to tackle object scale variation in dense prediction tasks. Although existing feature pyramid networks have greatly advanced visual recognition, inherent design defects inhibit them from capturing discriminative features and recognizing small objects. In this work, we propose Asymptotic Content-Aware Pyramid Attention Network (A3-FPN), to augment multi-scale feature representation via the asymptotically disentangled framework and content-aware attention modules. Specifically, A3-FPN employs a horizontally-spread column network that enables asymptotically global feature interaction and disentangles each level from all hierarchical representations. In feature fusion, it collects supplementary content from the adjacent level to generate position-wise offsets and weights for context-aware resampling, and learns deep context reweights to improve intra-category similarity. In feature reassembly, it further strengthens intra-scale discriminative feature learning and reassembles redundant features based on information content and spatial variation of feature maps. Extensive experiments on MS COCO, VisDrone2019-DET and Cityscapes demonstrate that A3-FPN can be easily integrated into state-of-the-art CNN and Transformer-based architectures, yielding remarkable performance gains. Notably, when paired with OneFormer and Swin-L backbone, A3-FPN achieves 49.6 mask AP on MS COCO and 85.6 mIoU on Cityscapes. Codes are available at https://github.com/mason-ching/A3-FPN.

Figures & tables

Figure 1: Illustration of FPN [ 29 ] and PAFPN [ 33 ] . (a) Top-down multi-scale feature fusion path in FPN. (b) Extra bottom-up path aggregation in PAFPN. Both methods have some defects: (1) information loss, (2) context-agnostic sampling, (3) pattern inconsistency.

Figure 2: (a) Average precision, parameters, and FLOPs of various feature pyramid networks evaluated on COCO val2017 [ 31 ] . Bubble area scales with model GFLOPs; (b) Inference latency vs. performance on COCO val2017 for feature pyramid models. All models are trained for 12 epochs using Faster R-CNN with ResNet-50 as the baseline. Inference latency is measured on a single NVIDIA RTX 4090 GPU.

Figure 3: Overall architecture of $A^{3}$ -FPN. (a) The bottom-up asymptotically disentangled fusion framework consisting of $m$ columns; (b) Multi-scale Context-aware Attention module for feature fusion; (c) Intra-scale Content-aware Attention module for feature reassembly.

Figure 4: Offset generator and context weight generator in multi-scale context-aware attention module. (a) Offset generator gathers context information to produce position-wise coordinate offset maps and sampling weight maps for the subsequent Resampler; (b) Context weight generator learns the relationship among different representation patterns and assigns the corresponding context weight to different-level features.

Figure 5: Comparison of different multi-scale designs, including (a) FPN [ 29 ] (layer-wise framework), (b) Gold-YOLO [ 47 ] (global convolutional framework), (c) Deformable Attention [ 65 ] (global attention framework), (d)

A^{3}

-FPN (asymptotically disentangled framework).

Figure 6: In first row, $H/8\times W/8$ feature maps are upsampled to $H/4\times W/4$ by bilinesar interpolation, DySample [ 34 ] and Cotext-aware Resampler (ours); In second row, we downsample $H/4\times W/4$ feature maps to $H/8\times W/8$ through strided convolution, CARAFE++ [ 49 ] and Cotext-aware Resampler (ours).

Figure 7: Visualization of detection results, the corresponding feature maps and heatmaps. Resampler refines coarsely sampled features and diminishes object displacement. The context weight generator learns the significance relationship of different feature patterns, decreasing the misclassifications and missed detections. ICAtten further enhances the discriminative features and alleviates complex background interference.

Method	Backbone	AP	AP 50	AP 75	AP S	AP M	AP L	Params(M)	FLOPs(G)
FPN [ 29 ]	R50	37.4	57.3	40.3	18.4	41.7	52.7	41.7	187
PConv † [ 51 ]	R50	38.5	59.9	41.4	-	-	-	42.8	-
FreqFusion [ 4 ]	R50	39.4	60.9	42.7	23.0	43.3	50.9	-	-
PAFPN [ 33 ]	R50	38.1	58.1	41.3	19.1	42.5	54.0	45.2	209
NAS-FPN † [ 16 ]	R50	37.7	54.5	41.1	15.5	44.5	56.9	68.2	366
AugFPN [ 17 ]	R50	38.7	61.2	41.9	24.0	42.5	49.5	–	–

Table 1: Performance of Faster RCNN [ 41 ] with different feature pyramid networks on MS COCO val2017 . † indicates model performance referred from other works.

Method	Backbone	Epoch	AP	AP 50	AP 75	AP S	AP M	AP L
RetinaNet [ 30 ]	R50	12	18.1	31.1	18.3	8.8	28.5	38.0
FPN [ 29 ]	R50	12	21.0	36.4	21.4	10.9	34.3	40.1
PAFPN [ 33 ]	R50	12	21.2	36.5	21.6	10.9	34.6	41.1
AugFPN [ 17 ]	R50	12	21.7	37.1	22.2	11.1	35.4	40.4
FPT [ 58 ]	R50	12	19.3	33.3	19.2	9.4	30.0	38.9
RCFPN [ 66 ]	R50	12	21.0	36.0	21.3	10.5	34.8	38.1

Table 2: Small object detection performance with different feature pyramid networks on the VisDrone2019-DET dataset. The baseline model is RetinaNet [ 30 ] .

Figure 8: Qualitative evaluation of various feature pyramid networks for object detection on MS COCO validation set, including FPN [ 29 ] , PAFPN [ 33 ] , NAS-FPN [ 16 ] , AFPN [ 57 ] and our $A^{3}$ -FPN. Odd rows are the detection results, and the others are the corresponding AblationCAM [ 40 ] visualization. The object category in the images is sheep.

Type	Method	Backbone	Epoch	AP	AP 50	AP 75	AP S	AP M	AP L
multi-stage CNN detectors	Cascade R-CNN [ 1 ]	R101	18	42.8	62.1	46.3	23.7	45.5	55.2
	$A^{3}$ -FPN-Lite	R101	18	44.3	62.3	47.6	25.3	47.1	57.6
	$A^{3}$ -FPN	R101	18	45.1	62.9	48.5	26.2	47.9	58.1
	RepPointsV2 [ 6 ]	R101	24	46.0	65.3	49.5	27.4	48.9	57.3
	$A^{3}$ -FPN-Lite	R101	24	47.3	66.1	50.4	27.9	49.6	58.4
	$A^{3}$ -FPN	R101	24	48.2	67.2	51.1	28.5	50.9	60.3

Table 3: Performance comparison between state-of-the-art detectors with and without incorporating

A^{3}

-FPN on MS COCO dataset. CNN detectors are evaluated on COCO test-dev , while vision transformer detectors are tested on COCO val2017 .

Type	Method	Backbone	Epoch	AP	AP 50	AP 75	AP S	AP M	AP L
CNN methods	FPN [ 29 ]	R50	12	34.7	55.7	37.2	18.3	37.4	47.2
	FPT [ 58 ]	R50	12	36.8	55.9	38.6	18.8	35.3	54.2
	CARAFE [ 49 ]	R50	12	35.4	56.7	37.6	16.9	38.1	51.3
	DySample+ [ 34 ]	R50	12	35.7	57.3	38.2	17.3	38.2	51.8
	FreqFusion [ 4 ]	R50	12	36.0	57.9	38.1	17.9	39.0	52.3
	$A^{2}$ -FPN [ 21 ]	R50	12	36.2	58.4	38.1	20.1	39.2	49.6

Table 4: Mask AP comparison of different instance segmentation models on COCO val2017 . CNN methods adopt Mask R-CNN [ 19 ] as the baseline architecture.

{}^{\textbf{*}}

denotes mask AP is evaluated on instance ground truths derived from panoptic annotations [ 23 ] and † indicates re-implementation results.

Figure 9: Qualitative evaluation. (a) Instance segmentation results of Mask RCNN [ 19 ] with different feature fusion approaches, including FPN [ 29 ] , FPT [ 58 ] , DySample [ 34 ] , CARAFE [ 49 ] and our $A^{3}$ -FPN; (b) and (d) are comparison between some unified transformer-based models (Mask2Former [ 7 ] and Mask DINO [ 25 ] ) with and without integrating $A^{3}$ -FPN on the instance and semantic segmentation task respectively; (c) Semantic segmentation visualization on Cityscapes validation set using different semantic segmentors, which are UperNet [ 54 ] , SegNext [ 18 ] , SegFormer [ 55 ] , SETR [ 64 ] , and our $A^{3}$ -FPN. Additional qualitative results are presented in Appendix E and F.

Type	Method	Backbone	Crop Size	Schedule	mIoU (s.s.)	mIoU (m.s.)
CNN segmentors	PointRend [ 24 ]	R50	512 $\times$ 1024	80k	76.47	78.13
	FaPN †† [ 22 ]	R50	512 $\times$ 1024	80k	79.07	-
	$A^{3}$ -FPN-Lite	R50	512 $\times$ 1024	80k	78.98	80.48
	$A^{3}$ -FPN	R50	512 $\times$ 1024	80k	79.93	81.54
	PSPNet [ 61 ]	R50	512 $\times$ 1024	80k	78.55	79.79
	PSANet [ 62 ]	R50	512 $\times$ 1024	80k	77.24	78.69

Table 5: Comparison with recent state-of-the-art semantic segmentation methods on Cityscapes [ 9 ] validation set. We calculate mIoU to assess the model performance and intra-category consistency of the final predictions. †† indicates our re-implementation results, while s.s. and m.s. denotes single-scale and multi-scale training, respectively.

Framework	AP ${}^{\text{box}}$	AP ${}^{\text{mask}}$	mIoU
with FPN’s modules:
top-down framework [ 29 ]	38.2	34.7	-
top-down + bottom up framework [ 33 ]	38.7	35.1	-
Gather-and-Distribute framework [ 47 ]	39.2	35.4	-
top-down asymptotically disentangled framework	39.8	35.9	-
with $A^{3}$ -FPN’s modules:

Table 6: Ablation study for different feature fusion frameworks. AP

{}^{\text{box}}

and AP

{}^{\text{mask}}

are evaluated on MS COCO val2017 , while mIoU is calculated on Cityscapes.

Table 16

MCAtten		ICAtten	AP	AP 50	AP 75
context-aware resampler	context weight generator	ICAtten	AP	AP 50	AP 75
standard feature fusion			54.13	80.23	59.56
✓	✗	✗	55.47	82.43	60.97
✓	✓	✗	56.45	82.27	62.03
✓	✓	✓	57.08	82.56	62.13

Table 9: Ablation study for MCAtten and ICAtten. With Faster RCNN [ 41 ] as the detector, models are trained and evaluated on PASCAL VOC training set (2007trainval and 2012trainval) and voc2007test, respectively.

Panel of Table 10 — Table 10: APs on PASCAL VOC 2007 test set and mIoUs on Cityscapes validation set with different thresholds in ICAtten.

Panel of Table 11 — Table 11: APs of our method on PASCAL VOC 2007 test set with different channel numbers per group in the offset generator and ICAtten.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

\left\{X_{1}^{offset},\cdots,X_{i-1}^{offset},X_{i+1}^{offset},\cdots,X_{\textbf{min}}^{offset}\right\}\leftarrow\mathbf{OffsetGererator}(\left\{X_{1}^{samp},\cdots,X_{i},\cdots,X_{\textbf{min}}^{samp}\right\})

Appendix

Algorithm 1 Bottom-up Asymptotic Content-Aware Pyramid Attention Network

Figure 10: Illustration of A3-FPN with the top-down asymptotic disentangled framework.

Figure 11: (a) Visualization of coordinate offsets and the corresponding attention weight in the offset generator. (b) Fusion process visualization in MCAtten.

Figure 12: (a) Instance segmentation results of Mask RCNN He et al. [2017] with different feature fusion approaches, including FPN Lin et al. [2017a] , FPT Zhang et al. [2020] , DySample Liu et al. [2023b] , CARAFE Wang et al. [2019] and our $A^{3}$ -FPN. (b) Qualitative comparison between some unified transformer-based models (Mask2Former Cheng et al. [2022] and Mask DINO Li et al. [2023a] ) with and without integrating $A^{3}$ -FPN on the instance segmentation task.

Figure 13: (a) Visualization on Cityscapes validation set using different semantic segmentors, which are UperNet Xiao et al. [2018] , SegNext Guo et al. [2022] , SegFormer Xie et al. [2021] , SETR Zheng et al. [2021] , and our $A^{3}$ -FPN. (b) Qualitative evaluation of unified transformer-based models (Mask2Former Cheng et al. [2022] and Mask DINO Li et al. [2023a] ) with and without integrating $A^{3}$ -FPN on the semantic segmentation task.

Hyperparameter	$A^{3}$ -FPN	$A^{3}$ -FPN-Lite
Squeeze	[1, 2, 4, 4]	[1, 2, 4, 8]
Using Resampling	[True, True, True]	[False, False, True]
Compress Channels	[16, 16, 16, 32]	[16, 16, 16, 16]
GN Group	[16, 16, 16, 32]	[16, 16, 16, 16]
RepBlock Number	2	1
Expansion	4.0	2.0

Appendix

Table 12: Hyperparameters setting between

A^{3}

-FPN and

A^{3}

-FPN-Lite.

Dense Prediction

Dense prediction maps images or other structured inputs to spatially aligned outputs such as labels, depth, motion, or physical fields.

See all 31 papers

Apr 3, 2026

Hierarchical Awareness Adapters with Hybrid Pyramid Feature Fusion for Dense Depth Prediction

Monocular depth estimation from a single RGB image remains a fundamental challenge in computer vision due to inherent scale ambiguity and the absence of explicit geometric cues. Existing approaches typically rely on increasingly complex network architectures to regress depth maps, which escalates training costs and computational overhead without fully exploiting inter-pixel spatial dependencies. We propose a multilevel perceptual conditional random field (CRF) model built upon the Swin Transformer backbone that addresses these limitations through three synergistic innovations: (1) an adaptive hybrid pyramid feature fusion (HPF) strategy that captures both short-range and long-range dependencies by combining multi-scale spatial pyramid pooling with biaxial feature aggregation, enabling effective integration of global and local contextual information; (2) a hierarchical awareness adapter (HA) that enriches cross-level feature interactions within the encoder through lightweight broadcast modules with learnable dimensional scaling, reducing computational complexity while enhancing representational capacity; and (3) a fully-connected CRF decoder with dynamic scaling attention that models fine-grained pixel-level spatial relationships, incorporating a bias learning unit to prevent extreme-value collapse and ensure stable training. Extensive experiments on NYU Depth v2, KITTI, and MatterPort3D datasets demonstrate that our method achieves state-of-the-art performance, reducing Abs Rel to 0.088 (

-

7.4%) and RMSE to 0.316 (

-

5.4%) on NYU Depth v2, while attaining near-perfect threshold accuracy (

δ< 1.25^3 \approx 99.8\%

) on KITTI with only 194M parameters and 21ms inference time.

Wuqi Su, Huilun Song, Chen Zhao +1

Zhejiang Gongshang University

Monocular Depth Estimation Multi-Scale Feature Fusion Conditional Random Fields

Ask PDF

Jul 17, 2026

DPNeXt: A Lightweight Multi-Scale Feature Fusion Framework for Efficient ViT-Based Multi-Task Dense Prediction

Multi-Task Learning (MTL) in robotics perception systems supports comprehensive 3D spatial scene understanding by integrating semantic segmentation and depth estimation. While Vision Foundation Models (VFMs) are increasingly adopted as robust feature encoders, existing decoding strategies present a critical bottleneck. To address this, we propose DPNeXt, a streamlined multi-scale feature fusion decoder and efficient alternative to the standard Dense Prediction Transformer (DPT). DPNeXt uses dual depthwise separable inverted bottlenecks to improve frozen VFM utilization through fusion-centric decoding and independent task modularization. To further mitigate negative inductive transfer between tasks, we introduce the Multi-Task Boundary Guidance (MTBG) strategy. Unlike prior boundary-aware methods that add fusion modules or gating, MTBG applies symmetric boundary-focused supervision to encourage geometric consistency without extra annotation or inference cost. Experiments on Cityscapes show that DPNeXt-S outperforms prior state-of-the-art (SOTA) MTL models, while DPNeXt-B further improves the overall performance and achieves the best results among the compared methods. On NYUv2, DPNeXt-B also achieves the best semantic segmentation and depth estimation results among the compared methods while requiring substantially fewer trainable parameters than prior large-scale MTL models. Compared with the standard DPT, DPNeXt-S reduces trainable parameters by 78.6% and achieves the fastest inference speed among the compared models on resource-constrained laptop hardware. The source code, model checkpoints, and a demo video will be made available at https://github.com/kangjehun/DPNeXt.

Jehun Kang, Jungha Wang, Youngjun Hwang +1

School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, South Korea

Multi-Task Learning Multi-Scale Feature Fusion Efficient ViTs Robotic Perception

Ask PDF

Jun 12, 2026

Context-Guided Semantic Alignment for Feature Fusion Networks

Feature fusion networks are fundamental components in modern object detectors, aggregating multi-scale features to detect objects of varying sizes. However, directly fusing features from different pyramid levels often introduces semantic inconsistency due to their heterogeneous representations. In this paper, we propose Feature Interaction NEtwork (FINE), a lightweight semantic alignment module that refines low-level features via high-level contextual guidance using cross-level attention prior to fusion. To bridge the structural gap and ensure computational efficiency, we introduce an Alignment-Aware Token Sampling that aligns corresponding spatial regions across scales, reducing the attention complexity by an order of magnitude. The resulting attention weights generate a spatial-channel modulation map that is upsampled and applied to the low-level features via residual element-wise modulation. This mechanism ensures that the network selectively enhances semantically relevant pixels while preserving the sub-pixel localization accuracy necessary for dense prediction tasks. FINE is generally applicable to various detectors and consistently improves detection accuracy without compromising efficiency.

Hyungseop Lee, Jiho Lee, Woochul Kang

Department of Embedded Systems Engineering, Incheon National University, Yeonsu-gu 22012, South Korea

Multi-Scale Feature Fusion Object Detection Attention Mechanisms Multi-Scale Attention

Ask PDF

Abstract

Dense Prediction

Explore similar work

Hierarchical Awareness Adapters with Hybrid Pyramid Feature Fusion for Dense Depth Prediction

DPNeXt: A Lightweight Multi-Scale Feature Fusion Framework for Efficient ViT-Based Multi-Task Dense Prediction

Context-Guided Semantic Alignment for Feature Fusion Networks