ElasticFormer: Detecting Objects in HRW Shots via Elastic Computing Vision Transformer

Wenxi Li; Jingchen Huang; Chenyang Lyu; Moran Liu; Haozhe Lin; Guiguang Ding; Yuchen Guo

cs.CVOct 29, 2026

ElasticFormer: Detecting Objects in HRW Shots via Elastic Computing Vision Transformer

Authors: Wenxi Li, Jingchen Huang, Chenyang Lyu, Moran Liu, Haozhe Lin, Guiguang Ding, Yuchen Guo

Organizations: KLATASDS-MOE, School of Statistics, East China Normal University, Shanghai, China · Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing, China · Department of Automation, Tsinghua University, Beijing, China · AI Business, Alibaba International Digital Commerce, Hangzhou, China ·

Abstract

Recent advances in gigapixel-level imaging have brought High-Resolution Wide shots to the forefront of research. However, these images present significant challenges: extreme sparsity of foreground, gigapixel-level resolutions and diverse target counts. This makes traditional close-up detectors inaccurate and slow as they are overwhelmed by the background. Although previous research has explored sparse backbones, their fixed sparsity patterns lack the adaptability required to handle diverse target numbers. To address this, we introduce ElasticFormer, a sparse backbone that dynamically allocates computational resources based on foreground proportion. After scoring windows based on variance, proposed ElasticSelector module will predict the foreground proportion for top-k selection. The mechanism guides the model to select target-containing windows, scaling resources in areas where objects are clustered. We introduce a novel loss function combined with the 3-phase training strategy for ElasticSelector, allowing it to function properly when bounding box annotations are missing. A WSOD study is carried on PASCAL VOC 2007 to evaluate its extensibility. Further, ElasticNet is created to verify its backbone-agnostic nature. In experiments on the PANDA gigapixel benchmark, ElasticFormer reduces backbone FLOPs by 80% while achieving a significant improvement in AP50 when compared to fixed-ratio sparse methods.

Figures & tables

Figure 1: Comparison of inference region selection between different backbones. (b) SparseFormer selects a fixed ratio of regions for processing, regardless of input variability. (c) ElasticFormer adaptively selects a dynamic ratio of informative regions based on the input image content, enabling more flexible and efficient inference.

Figure 2: Architecture Overview of ElasticFormer. The green box represents the ElasticBlock,in which the process of “Scoring” is listed in Eq. 1 . Example scoring results of 9 windows is listed here, where the higher score corresponds to the darker color, indicating larger variance and a greater likelihood of containing targets. ElasticSelector selects windows according to their scores and the predicted $\hat{\rho}$ , shown in Eq. 3 . Here the sparse case select 2 windows out of 9 while the dense case select 8 windows as $\hat{\rho}$ differs. ElasticSelector is trained via $\mathcal{L}_{select}$ , in which $\rho$ represents the real foreground proportion.

Method	Backbone	GFLOPs-F	GFLOPs-B	GFLOPs-O	$\text{AP}_{50}$	$\text{AP}_{\text{small}}$	$\text{AP}_{\text{meduim}}$	$\text{AP}_{\text{large}}$
CascadeRCNN [ 3 ]	ResNet-101 [ 20 ]	15.54	295.24	310.78	-	0.227	0.579	0.765
ClusDet [ 38 ]	ResNet-50 [ 20 ]	10.35	196.71	207.07	0.718	0.219	0.696	0.782
DMNet [ 21 ]	ResNet-50	10.35	196.71	207.07	0.540	0.119	0.371	0.714
FasterRCNN [ 16 ]	ResNet-50	10.35	196.71	207.07	0.705	0.203	0.712	0.760
GigaDet [ 6 ]	CSP-DarkNet-53	4.61	87.59	92.20	0.684	0.210	0.599	0.762
PAN [ 16 ]	ResNet-50	10.35	196.71	207.07	0.715	0.256	0.719	0.768

Table 1: Computational cost and accuracy comparison of different models on PANDA. GFLOPs are calculated exclusively for the backbone and neck when applicable. “F”, “B” and “O” denote foreground, background and overall respectively.

Method	Backbone	GFLOPs-F	GFLOPs-B	GFLOPs-O	mAP(%)	CorLoc(%)
WSOD 2 [ 39 ] +GoogleNet	GoogleNet [ 34 ]	5.56	12.38	17.94	37.9	52.8
WSOD 2 +VGG16	VGG16 [ 30 ]	36.88	86.05	122.93	47.2	59.5
WSOD 2 +ResNet-50	ResNet-50 [ 20 ]	63.01	111.90	174.91	46.7	59.9
WSOD 2 +ResNet-101	ResNet-101 [ 20 ]	70.83	165.27	236.10	48.9	61.1
Modified-DINO+Swin-T	Swin-T [ 28 ]	11.49	26.80	38.29	49.9	63.7
Modified-DINO+SparseFormer	SparseFormer [ 22 ]	9.11	13.01	22.12	50.3	63.1

Table 2: Comparisons of FLOPs, mean average precision and correct localization on different backbones in WSOD tasks. We modified DINO’s loss function using a Multiple Instance Learning (MIL) based loss to make it suitable for the WSOD task.

Figure 3: AP 50 (%) vs. the length of Phase 1. As Phase 1 becomes longer, AP 50 first increases and then decreases with the maximum at 24 epochs.

Strategies	Supervision	Windows selection rates (%)				AP 50
Strategies	Supervision	Stage1	Stage2	Stage3	Stage4	AP 50
1-phase	GT	15.11	15.32	12.79	15.73	0.791
1-phase	Pseudo-GT	14.02	14.24	9.80	12.97	0.784
3-phase	GT	14.15	14.55	13.28	14.95	0.798
3-phase	Pseudo-GT	11.31	10.41	7.56	12.09	0.806

Table 3: Windows selection rates and AP 50 across different configures and stages. Pseudo-GT means outputs from the detection head. Rates on Stage 3 is always the lowest.

Figure 4: Visualization of scored windows. We illustrate scored windows from each stage of ElasticFormer, with the top two rows from PANDA [ 2 ] and the others from PASCAL VOC 2007 [ 1 ] for WSOD task. Highlighted windows indicate regions requiring further feature extraction.

$\alpha$	$\gamma$	AP 50	GFLOPs-O	GFLOPs-F	F-ratio (%)
1.0	0.1	0.802	14.98	6.84	45.7
	0.2	0.799	13.17	6.00	45.6
	0.3	0.793	11.44	5.25	45.9
0.8	0.1	0.806	13.13	7.02	53.5
	0.2	0.801	11.91	6.32	53.1
	0.3	0.795	10.28	5.48	53.3

Table 4: AP 50 and FLOPs across different

\alpha

and

\gamma

. F-ratio is the proportion of foreground computation to the total, indicating a backbone’s attention to foreground regions.

Scoring Mechanism	AP 50	GFLOPs-F	F-ratio (%)	Latency (ms)
Variance-based(Ours)	0.806	7.02	53.5	0.397
Small objectness	0.803	7.14	53.8	1.213
Gradient density	0.798	5.74	41.7	0.829

Table 5: AP 50 , FLOPs and latency across different scoring machanism. Latency is measured as the average over four stages, 100 forward passes of the backbone.

Figure 5: AP 50 vs. sparsity rate on SparseFormer and ElasticFormer. The annotated data in the figure is obtained through linear interpolation.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 1 — Figure 1: Comparison of the loss for different strategies. We demonstrate 4 loss curves in each sub-figure, representing the loss for each stage in ElasticFormer.

Figure 2: Visualization of scored windows. Left two columns come from PANDA [ 2 ] and right two columns come from PASCAL VOC 2007 [ 1 ] .

Object Detection

Object detection locates and classifies objects in images or video, balancing localization accuracy, robustness, and inference speed.

See all 629 papers

BiHashFormer: Hash-Driven Dual-Branch Transformer for Efficient Object Detection in HRW Shots
Oct 29, 2026Jingchen Huang, Wenxi Li, Chenyang Lyu +3Object Detection Efficient Object Detection
AI Business, Alibaba International Digital Commerce, Hangzhou, China · Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing, China
Learnable Token Sparsification for Efficient Gigapixel Whole Slide Image Reasoning
Jun 7, 2026Jingzhi Chen, Landi He, Zhuo Chen +2Efficient VLM Inference Medical VLMs
Shenzhen University of Advanced Technology
UHR-DETR: Efficient End-to-End Small Object Detection for Ultra-High-Resolution Remote Sensing Imagery
Apr 23, 2026Jingfang Li, Haoran Zhu, Wen Yang +4Remote Sensing Small Object Detection Small Object Detection
School of Electronic Information, Wuhan University, Wuhan, 430072 China

Oct 29, 2026J/K move · Enter open · S save

BiHashFormer: Hash-Driven Dual-Branch Transformer for Efficient Object Detection in HRW Shots

Jingchen Huang, Wenxi Li, Chenyang Lyu, Moran Liu+2

AI Business, Alibaba International Digital Commerce, Hangzhou, China · Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing, China

Object Detection Efficient Object Detection Detection Transformer Efficient Attention

Open paper AskPDF arXiv

Recent advances in gigapixel-level imaging have brought High-Resolution Wide shots to the forefront of research. However, these images present significant challenges: extreme sparsity of foreground, gigapixel-level resolutions and interleaving of foreground and background. This causes traditional detectors and attention mechanisms to be hindered by background, resulting in inefficiency and inaccuracy. To tackle this problem, we propose BiHashFormer, a dual sparsified branch transformer built on hashing. RoI Selector will predict the foreground proportion for top-k selecting target-containing windows. The compression branch processes windows selected and refines attention into HashAttention by discarding half of the key-value pairs, reducing cost and improving efficiency. The compensation branch uses HashMiner to perform low-cost hash searches in the remaining regions, recovering features of missed objects. Experiments confirm that the selection relationship in HashAttention is one-way and reveal the preference of different queries for keys. In experiments on the gigapixel benchmark PANDA, BiHashFormer reduces 37.5% backbone FLOPs while improving AP50 to 81.0%. Moreover, this parallel dual-branch structure allows the model to avoid inference delays caused by inter-branch dependencies, while achieving much greater stability than single-branch models, specially with low rates of RoI selection.

Abstract

Object Detection

Explore similar work

BiHashFormer: Hash-Driven Dual-Branch Transformer for Efficient Object Detection in HRW Shots

Learnable Token Sparsification for Efficient Gigapixel Whole Slide Image Reasoning

UHR-DETR: Efficient End-to-End Small Object Detection for Ultra-High-Resolution Remote Sensing Imagery