BiHashFormer: Hash-Driven Dual-Branch Transformer for Efficient Object Detection in HRW Shots

Jingchen Huang; Wenxi Li; Chenyang Lyu; Moran Liu; Haozhe Lin; Yuchen Guo

cs.CVOct 29, 2026

BiHashFormer: Hash-Driven Dual-Branch Transformer for Efficient Object Detection in HRW Shots

Authors: Jingchen Huang, Wenxi Li, Chenyang Lyu, Moran Liu, Haozhe Lin, Yuchen Guo

Organizations: AI Business, Alibaba International Digital Commerce, Hangzhou, China · Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing, China

Abstract

Recent advances in gigapixel-level imaging have brought High-Resolution Wide shots to the forefront of research. However, these images present significant challenges: extreme sparsity of foreground, gigapixel-level resolutions and interleaving of foreground and background. This causes traditional detectors and attention mechanisms to be hindered by background, resulting in inefficiency and inaccuracy. To tackle this problem, we propose BiHashFormer, a dual sparsified branch transformer built on hashing. RoI Selector will predict the foreground proportion for top-k selecting target-containing windows. The compression branch processes windows selected and refines attention into HashAttention by discarding half of the key-value pairs, reducing cost and improving efficiency. The compensation branch uses HashMiner to perform low-cost hash searches in the remaining regions, recovering features of missed objects. Experiments confirm that the selection relationship in HashAttention is one-way and reveal the preference of different queries for keys. In experiments on the gigapixel benchmark PANDA, BiHashFormer reduces 37.5% backbone FLOPs while improving AP50 to 81.0%. Moreover, this parallel dual-branch structure allows the model to avoid inference delays caused by inter-branch dependencies, while achieving much greater stability than single-branch models, specially with low rates of RoI selection.

Figures & tables

Figure 1: Comparison of RoI between different backbones. (b) SparseFormer selects a fixed ratio of regions for processing, regardless of input variability. (c) BiHashFormer combines lower selection ratio in the compression branch with dynamic compensation, making it more flexible and efficient. For the compensation branch, windows are drawn if any key in them is selected.

Figure 2: Architecture overview of BiHashFormer. CPR: C om PR ession branch. CPS: C om P en S ation branch. RoI Selector devides images into RoI for CPR and Non-RoI for CPS. Hash Code in the figure demonstrate an example of 2-bit for Eq. ( 9 ). An example of HashAttention is listed in the green box: for the red query, pixels in the image are selected by KV Selector. Prototype codes are trained in CPR through $\mathcal{L}_{proto}$ , then used in HashMiner. HashMiner selects tokens for further feature extraction in CPS.

Method	Backbone	GFLOPs			AP
		F	B	O	50	small	medium	large
CascadeRCNN	ResNet-101	15.54	295.24	310.78	-	0.227	0.579	0.765
ClusDet	ResNet-50	10.35	196.71	207.07	0.718	0.219	0.696	0.782
DMNet	ResNet-50	10.35	196.71	207.07	0.540	0.119	0.371	0.714
FasterRCNN	ResNet-50	10.35	196.71	207.07	0.705	0.203	0.712	0.760
GigaDet	CSP-DarkNet-53	4.61	87.59	92.20	0.684	0.210	0.599	0.762

Table 1: Computational cost and accuracy comparison of different models on PANDA. GFLOPs are calculated exclusively for the backbone and neck when applicable. “F”, “B” and “O” denote foreground, background and overall respectively. Baseline sources: [ 1 , 14 , 31 , 15 , 3 , 3 , 18 , 15 , 6 , 7 , 4 , 14 ] .

Figure 4: AP 50 vs. windows selection ratio

\rho

on SparseFormer and BiHashFormer. The values annotated correspond to the

\text{AP}_{50}

\rho=0.147

Table 6

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Selection Method	MFLOPs	$\text{AP}_{\text{small}}$	Latency(ms)
Random	0	0.291	0.134
MLP	1803	0.375	1.793
Gradient	251	0.336	0.847
HashMiner (Ours)	1179	0.591	1.196

Appendix

Table X1 :

\text{AP}_{\text{small}}

, FLOPs and latency with different compensation methods. FLOPs: selection cost only. Latency is measured as the average over four stages, 50 forward passes of the backbone..

Window Size $w$	GFLOPs-F	GFLOPs-CPR	$\text{AP}_{50}$	FPS
7	6.11	8.92	0.792	1.425
14	6.89	22.21	0.806	0.811
21	7.30	38.12	0.810	0.356
28	7.17	73.86	0.803	0.174

Appendix

Table X2 :

\text{AP}_{50}

, FLOPs and FPS with different

w

. GFLOPs-CPR: compression branch only. FPS is evaluated on single RTX 4090.

Compensation Method	GFLOPs	$\text{AP}_{50}$	$\text{AP}_{\text{small}}$	Eff
-	0	0.786	0.287	-
Attention Module	13.28	0.793	0.311	0.18
CPS Branch (Ours)	8.29	0.810	0.591	3.67

Appendix

Table X3 : AP, FLOPs and Eff with different compensation methods. FLOPs: compensation cost only. AP denotes the performance of the complete model and Eff is defined in Equation (17).

$\gamma$	AP 50 $\uparrow$	GFLOPs-O $\downarrow$	GFLOPs-F $\downarrow$	FPS $\uparrow$
0.1	0.810	47.31	7.30	0.356
0.3	0.808	44.25	6.91	0.381
0.5	0.807	42.57	6.72	0.397
0.7	0.804	40.86	6.19	0.414
0.9	0.801	37.59	5.68	0.452

Appendix

Table X4 : AP 50 , FLOPs and FPS across different

\gamma

Method	AP 50	Latency(s)
PAN [ 3 ]	0.715	44.2
RT-DETR [ 15 ]	0.612	22.5
DynamicDet [ 6 ]	0.639	17.3
DINO+SparseFormer [ 4 ]	0.780	13.9
DINO [ 14 ] +Ours	0.810	11.1

Appendix

Table X5 : AGX Orin

Method	AP 50	Latency(s)
PAN [ 3 ]	0.715	10.9
RT-DETR [ 15 ]	0.612	4.2
DynamicDet [ 6 ]	0.639	3.4
DINO+SparseFormer [ 4 ]	0.780	3.5
DINO [ 14 ] +Ours	0.810	2.8

Appendix

Table X6 : RTX 4090

Method	mAP	GFLOPs
DINO+Efficientvit [ 8 ]	74.39	159.24
vHeat-T [ 11 ]	77.82	243.18
Swin-T [ 7 ]	77.63	221.32
DINO+SparseFormer [ 4 ]	77.94	174.31
DINO [ 14 ] +Ours	79.52	89.63

Appendix

Table X7 : DOTA

Method	AP 50	AR 100
DINO+Efficientvit [ 8 ]	0.473	0.338
vHeat-T [ 11 ]	0.512	0.385
Swin-T [ 7 ]	0.481	0.337
DINO+SparseFormer [ 4 ]	0.554	0.408
DINO [ 14 ] +Ours	0.602	0.452

Appendix

Table X8 : VisDrone

Backbone	Prototype codes	AP	$\mathrm{AP}_{50}$	$\mathrm{AP}_{\mathrm{small}}$	GFLOPs O	GFLOPs CPS
Swin-T	–	46.0	68.1	31.2	138.34	–
BiHashFormer	1	47.0	70.2	32.4	86.96	5.01
BiHashFormer	3	48.3	71.5	34.8	89.04	7.08
BiHashFormer	5	47.2	70.8	33.1	96.83	14.88

Appendix

Table X9 : Prototype-number ablation and comparison with Swin-T on COCO. AP metrics are percentages. GFLOPs-O covers the backbone and neck; GFLOPs-CPS covers the compensation branch. Swin-T accuracy follows the

3\times

Mask R-CNN results in [ 7 ] .

Configuration	GFLOPs-O $\downarrow$	Latency (s) $\downarrow$	$\mathrm{AP}_{50}$ $\uparrow$
Linformer + DINO	57.48	3.15	0.792
XAttention + DINO	64.93	3.25	0.799
SparseFormer + DINO	75.71	3.5	0.780
BiHashFormer + DINO	47.31	2.8	0.810

Appendix

Table X10 : Comparison of complete model configurations on PANDA with the same DINO head. GFLOPs-O covers the backbone and neck. Latency is in seconds, and

\mathrm{AP}_{50}

is reported on the 0–1 scale.

Backbone	GFLOPs F	GFLOPs B	GFLOPs O	$\mathrm{AP}_{50}$	$\mathrm{AP}_{\mathrm{small}}$
DINOv3 ViT-S+/16	23.61	448.71	472.32	0.768	0.411
BiHash-DINOv3 ViT-S+/16	25.96	142.25	168.21	0.808	0.527

Appendix

Table X11 : PANDA accuracy and computation with identical DINOv3 initialization. Both models use the same ViT-S+/16 checkpoint, detector, and 36-epoch protocol. F, B, and O denote foreground, background, and overall backbone/neck computation. AP metrics use the 0–1 scale.

Backbone	Latency (ms) $\downarrow$	FPS $\uparrow$
DINOv3 ViT-S+/16	13,280	0.075
BiHash-DINOv3 ViT-S+/16	5,780	0.173

Appendix

Table X12 : PANDA wall-clock efficiency with DINOv3 pretraining. Measurements use one RTX 4090.

Method	Time/epoch (min)	Total time (h)	Epoch to 0.75 $\mathrm{AP}_{50}$	Time to 0.75 $\mathrm{AP}_{50}$ (h)	Final $\mathrm{AP}_{50}$
DINO + SparseFormer	38.7	23.2	28	18.0	0.780
DINO + BiHashFormer	32.8	19.7	23	12.6	0.810

Appendix

Table X13 : Training time and convergence on PANDA under the same 36-epoch protocol. Both models use eight RTX 4090 GPUs and one image per GPU. AP metrics use the 0–1 scale.

Figure X1 : Enlarged detail from the architecture overview in Figure 2. The vector crop shows the HashAttention and HashMiner pathways, prototype codes, and their connections.

Figure X2 : Enlarged view of the query–key visualization in Figure 3. The original figure content is unchanged.

Object Detection

Object detection locates and classifies objects in images or video, balancing localization accuracy, robustness, and inference speed.

See all 629 papers

ElasticFormer: Detecting Objects in HRW Shots via Elastic Computing Vision Transformer
Oct 29, 2026Wenxi Li, Jingchen Huang, Chenyang Lyu +4Object Detection Efficient Object Detection
KLATASDS-MOE, School of Statistics, East China Normal University, Shanghai, China · Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing, China · Department of Automation, Tsinghua University, Beijing, China +3
Beyond Post-Quantization: Native Hash Learning with a Dedicated HASH Token
Jul 3, 2026Xinze Liu, Ding Wang, Hengjie Zhu +1Vision Transformer Image Retrieval
Institute of Information Engineering, CAS, Beijing 100190, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing 100190, China · Department of Applied Mathematics and Statistics, Johns Hopkins University, Baltimore, Maryland 21218, USA
ARTA: Adaptive Mixed-Resolution Token Allocation for Efficient Dense Feature Extraction
Mar 27, 2026David Hagerman, Roman Naeem, Erik Brorsson +2Efficient ViTs Vision Transformer
Chalmers University of TechnologyChalmers University of technology, 412 96 Gothenburg, Sweden

Oct 29, 2026J/K move · Enter open · S save

ElasticFormer: Detecting Objects in HRW Shots via Elastic Computing Vision Transformer

Wenxi Li, Jingchen Huang, Chenyang Lyu, Moran Liu+3

KLATASDS-MOE, School of Statistics, East China Normal University, Shanghai, China · Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing, China · Department of Automation, Tsinghua University, Beijing, China+3

Object Detection Efficient Object Detection Efficient ViTs Vision Transformer

Open paper AskPDF arXiv

Recent advances in gigapixel-level imaging have brought High-Resolution Wide shots to the forefront of research. However, these images present significant challenges: extreme sparsity of foreground, gigapixel-level resolutions and diverse target counts. This makes traditional close-up detectors inaccurate and slow as they are overwhelmed by the background. Although previous research has explored sparse backbones, their fixed sparsity patterns lack the adaptability required to handle diverse target numbers. To address this, we introduce ElasticFormer, a sparse backbone that dynamically allocates computational resources based on foreground proportion. After scoring windows based on variance, proposed ElasticSelector module will predict the foreground proportion for top-k selection. The mechanism guides the model to select target-containing windows, scaling resources in areas where objects are clustered. We introduce a novel loss function combined with the 3-phase training strategy for ElasticSelector, allowing it to function properly when bounding box annotations are missing. A WSOD study is carried on PASCAL VOC 2007 to evaluate its extensibility. Further, ElasticNet is created to verify its backbone-agnostic nature. In experiments on the PANDA gigapixel benchmark, ElasticFormer reduces backbone FLOPs by 80% while achieving a significant improvement in AP50 when compared to fixed-ratio sparse methods.

Abstract

Object Detection

Explore similar work

ElasticFormer: Detecting Objects in HRW Shots via Elastic Computing Vision Transformer

Beyond Post-Quantization: Native Hash Learning with a Dedicated HASH Token

ARTA: Adaptive Mixed-Resolution Token Allocation for Efficient Dense Feature Extraction