FUSEye: Training-Light Fisheye Detection with Overlapping Views and Zero-Initialized Adapters

Wenya Su; Kai Luo; Di Wen; Ruiping Liu; Yufan Chen; Junwei Zheng; Kunyu Peng; Kailun Yang

cs.CVOct 2, 2026

FUSEye: Training-Light Fisheye Detection with Overlapping Views and Zero-Initialized Adapters

Authors: Wenya Su, Kai Luo, Di Wen, Ruiping Liu, Yufan Chen, Junwei Zheng, Kunyu Peng, Kailun Yang

Organizations: School of Artificial Intelligence and Robotics and the National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, China · Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology, Germany

Abstract

Fisheye cameras give mobile robots a single-sensor, low-cost view of their surroundings, yet the COCO-pretrained detectors that practitioners routinely reuse fail on them: strong radial distortion warps local image structure, while boundary compression shrinks objects to near-invisible sizes. Full fine-tuning closes much of the gap but requires abundant fisheye labels and compute. We present FUSEye, a training-light framework that turns a frozen-backbone COCO-pretrained extra-large YOLO26 detector (YOLO26-x) into a fisheye detector. FUSEye adds roughly 227k new parameters while updating the inserted modules and the pretrained detection head. It addresses the transfer gap at three causally linked levels. At the input level, overlapping grid view generation and box remapping (GridViews) enlarge compressed boundary regions. At the feature level, zero-initialized residual adapters (Z-Adapters) correct distortion-induced feature misalignment. At the decision level, learned cross-projection agreement fusion (AgreeFusion) promotes low-confidence detections only when they are supported by consistent evidence across multiple views. On the WoodScape surround-view fisheye benchmark, FUSEye raises YOLO26-x from 0.148 to 0.266 mAP50 and retains 84.3% fully fine-tuned accuracy. Moreover, randomly using only 25% of the labeled training images, FUSEye achieves 0.2597 mAP50, retaining 97.6% of its full-label performance. FUSEye also consistently improves YOLOv8-11 detectors, showing that the recipe is architecture-agnostic. Source code will be available at https://github.com/Su-wenya/FUSEye.

Figures & tables

Fig. 1: Overall architecture of FUSEye. FUSEye adapts a COCO-pretrained detector to fisheye imagery while keeping its backbone weights frozen. It combines overlapping-view inference, residual feature adaptation, and detection-level agreement fusion. All views share the same adapted detector, and their predictions are remapped to a common coordinate system before fusion. The adapters and detection head are updated during detector adaptation, while a lightweight scorer learns to re-score the resulting multi-view clusters.

Fig. 2: Detailed designs of the three FUSEye modules. (a) GridViews generates the full view and four $857\times 647$ corner crops from a $1280\times 966$ image. Adjacent crops overlap by $423$ pixels horizontally and $319$ pixels vertically. All views are resized to $640\times 640$ , and their detections are remapped to the original image coordinates. (b) Zero-initialized residual Z-Adapters are inserted after P4, P5, and C2PSA to model local distortion while preserving the pretrained features. (c) AgreeFusion clusters aligned boxes by class and IoU and uses learned agreement features to re-score and fuse multi-view detections, followed by class-wise non-maximum suppression.

Model	Variant	Backbone blocks	Overall		Per-class $\mathrm{AP}_{50{:}95}$
Model	Variant	Backbone blocks	$\mathrm{AP}_{50}\uparrow$	$\mathrm{AP}_{50{:}95}\uparrow$	Car $\uparrow$	Person $\uparrow$	Bus $\uparrow$
YOLOv8 [ 37 ]	Nano	C2f	5.44	3.94	6.66	5.15	0.00
	Small		9.12	6.26	10.37	8.40	0.00
	Medium		12.71	8.95	13.47	12.39	0.99
	Large		13.07	9.66	14.25	13.73	0.99
	Extra-large		12.67	9.34	14.14	13.76	0.12

TABLE 1: Direct-transfer results of COCO-pretrained detectors on WoodScape MVR [ 1 ] . Results are reported as percentages. Backbone blocks list the main feature-extraction components; slashes separate component types. Bold and underline denote the best and second-best distinct results, respectively; ties receive the same formatting.

Method	Backbone frozen	Overall		Per-class $\mathrm{AP}_{50{:}95}$
Method	Backbone frozen	$\mathrm{AP}_{50}\uparrow$	$\mathrm{AP}_{50{:}95}\uparrow$	Car $\uparrow$	Person $\uparrow$	Bus $\uparrow$
Direct transfer	Yes	14.80	10.96	15.74	15.67	1.49
FUSEye	Yes	26.61	17.26	26.83	24.06	0.87
Full fine-tuning	No	31.56	22.82	42.48	22.44	3.54

TABLE 2: Comparison of adaptation strategies for YOLO26-x on WoodScape MVR. Bold and underline denote the best and second-best results, respectively.

GV	ZA	AF	Overall		Per-class $\mathrm{AP}_{50{:}95}$
GV	ZA	AF	$\mathrm{AP}_{50}\uparrow$	$\mathrm{AP}_{50{:}95}\uparrow$	Car $\uparrow$	Person $\uparrow$	Bus $\uparrow$
–	–	–	14.62	11.05	15.63	16.27	1.25
✓	–	–	15.49	11.22	13.60	18.51	1.55
–	✓	–	17.75	13.01	20.37	17.76	0.89
✓	✓	–	20.39	14.03	19.74	21.45	0.89
✓	–	✓	18.91	12.95	17.36	20.43	1.06

TABLE 3: Component ablation on YOLO26-x. GV, ZA, and AF denote GridViews, Z-Adapters, and AgreeFusion, respectively; the first row disables all three components. Bold and underline denote the best and second-best distinct results, respectively.

Metric	25% training set				100%
Metric	Subset 1	Subset 2	Subset 3	Mean $\pm$ SD	Full
$\mathrm{AP}_{50}\uparrow$	26.21	26.12	25.59	$25.97\pm 0.34$	26.61
$\mathrm{AP}_{50{:}95}\uparrow$	17.04	16.91	16.56	$16.84\pm 0.25$	17.26
$\mathrm{AP}^{\mathrm{car}}_{50{:}95}\uparrow$	26.21	26.19	26.11	$26.17\pm 0.05$	26.83
$\mathrm{AP}^{\mathrm{person}}_{50{:}95}\uparrow$	23.10	23.04	22.24	$22.79\pm 0.48$	24.06
$\mathrm{AP}^{\mathrm{bus}}_{50{:}95}\uparrow$	1.82	1.48	1.33	$1.55\pm 0.25$	0.87

TABLE 4: Results with

25\%

and

100\%

of the training set. Each

25\%

subset contains

1,520

images; mean

\pm

sample standard deviation is computed over three subsets.

Model	Method	Overall		Per-class $\mathrm{AP}_{50{:}95}$
Model	Method	$\mathrm{AP}_{50}\uparrow$	$\mathrm{AP}_{50{:}95}\uparrow$	Car $\uparrow$	Person $\uparrow$	Bus $\uparrow$
YOLOv8-x [ 37 ]	Base	11.89	9.05	12.79	13.36	0.99
YOLOv8-x [ 37 ]	FUSEye	17.69	11.50	17.70	16.35	0.45
YOLOv9-e [ 18 ]	Base	13.84	10.23	14.11	15.43	1.16
YOLOv9-e [ 18 ]	FUSEye	22.13	13.89	19.95	20.90	0.83
YOLOv10-x [ 19 ]	Base	12.25	9.16	13.40	13.09	0.99

TABLE 5: Cross-model evaluation on WoodScape MVR using the protocol in Sec. IV-A . The evaluated variants are Extra-large for YOLOv8, YOLOv10, and YOLO11, and Extended for YOLOv9. Base denotes COCO-pretrained direct transfer. Results are percentages; bold marks the better result in each pair.

Fig. 3: C2PSA activation maps from the YOLO26-x backbone for direct transfer and FUSEye. Warmer colors indicate stronger responses.

Fig. 4: Columns show ground truth, zero-shot COCO baseline, and complete FUSEye; insets enlarge representative detections on WoodScape MVR.

Object Detection

Object detection locates and classifies objects in images or video, balancing localization accuracy, robustness, and inference speed.

See all 624 papers

FishRoPE: Projective Rotary Position Embeddings for Omnidirectional Visual Perception
Apr 12, 2026Rahul Ahuja, Mudit Jain, Bala Murali Manoghar Sai Sudhakar +4Rotary Positional Embeddings Vision Foundation Model Adaptation
Automated Driving, Qualcomm Technologies, Inc · Automated Driving, Qualcomm India Private Limited
From Perspective to Fisheye Depth Estimation and Open-Vocabulary Segmentation
Aug 28, 2026Rit Gangopadhyay, Alex WongVision Foundation Model Adaptation Depth Estimation
Yale Vision Laboratory, Yale University, New Haven, CT 06520, USA
YOLOv14: Adaptive Real-Time Object Detection for Diverse Imaging Conditions
Aug 5, 2026Jinling Jia, Jian Lu, Jone Yawl +1Domain Adaptation Efficient Object Detection
Nanjing University of Posts and Telecommunications

Apr 12, 2026J/K move · Enter open · S save

FishRoPE: Projective Rotary Position Embeddings for Omnidirectional Visual Perception

Rahul Ahuja, Mudit Jain, Bala Murali Manoghar Sai Sudhakar, Venkatraman Narayanan+3

Automated Driving, Qualcomm Technologies, Inc · Automated Driving, Qualcomm India Private Limited

Rotary Positional Embeddings Vision Foundation Model Adaptation

Open paper AskPDF arXiv

Vision foundation models (VFMs) and Bird's Eye View (BEV) representation have advanced visual perception substantially, yet their internal spatial representations assume the rectilinear geometry of pinhole cameras. Fisheye cameras, widely deployed on production autonomous vehicles for their surround-view coverage, exhibit severe radial distortion that renders these representations geometrically inconsistent. At the same time, the scarcity of large-scale fisheye annotations makes retraining foundation models from scratch impractical. We present \ours, a lightweight framework that adapts frozen VFMs to fisheye geometry through two components: a frozen DINOv2 backbone with Low-Rank Adaptation (LoRA) that transfers rich self-supervised features to fisheye without task-specific pretraining, and Fisheye Rotary Position Embedding (FishRoPE), which reparameterizes the attention mechanism in the spherical coordinates of the fisheye projection so that both self-attention and cross-attention operate on angular separation rather than pixel distance. FishRoPE is architecture-agnostic, introduces negligible computational overhead, and naturally reduces to the standard formulation under pinhole geometry. We evaluate \ours on WoodScape 2D detection (54.3 mAP) and SynWoodScapes BEV segmentation (65.1 mIoU), where it achieves state-of-the-art results on both benchmarks.

Abstract

Object Detection

Explore similar work

FishRoPE: Projective Rotary Position Embeddings for Omnidirectional Visual Perception

From Perspective to Fisheye Depth Estimation and Open-Vocabulary Segmentation

YOLOv14: Adaptive Real-Time Object Detection for Diverse Imaging Conditions