Vision foundation models (VFMs) and Bird's Eye View (BEV) representation have advanced visual perception substantially, yet their internal spatial representations assume the rectilinear geometry of pinhole cameras. Fisheye cameras, widely deployed on production autonomous vehicles for their surround-view coverage, exhibit severe radial distortion that renders these representations geometrically inconsistent. At the same time, the scarcity of large-scale fisheye annotations makes retraining foundation models from scratch impractical. We present \ours, a lightweight framework that adapts frozen VFMs to fisheye geometry through two components: a frozen DINOv2 backbone with Low-Rank Adaptation (LoRA) that transfers rich self-supervised features to fisheye without task-specific pretraining, and Fisheye Rotary Position Embedding (FishRoPE), which reparameterizes the attention mechanism in the spherical coordinates of the fisheye projection so that both self-attention and cross-attention operate on angular separation rather than pixel distance. FishRoPE is architecture-agnostic, introduces negligible computational overhead, and naturally reduces to the standard formulation under pinhole geometry. We evaluate \ours on WoodScape 2D detection (54.3 mAP) and SynWoodScapes BEV segmentation (65.1 mIoU), where it achieves state-of-the-art results on both benchmarks.
Figures & tables
Figure 1: Architecture overview. FishRoPE comprises (1) a frozen DINOv2 backbone with LoRA adaptation for multi-scale feature extraction from fisheye images, (2) a FishRoPE-enhanced feature encoder that embeds fisheye-aware angular geometry into self-attention, and (3) task-specific heads for 2D detection and BEV segmentation via Kannala–Brandt view transformation.
Method
Backbone
Reported score ↑
FisheyeYOLO (24-sided polygon) [ 18 ]
R-18
44.65 †
FisheyeDetNet (polygon) [ 23 ]
R-18
49.5 ‡
FishRoPE (R-18 + FishRoPE)
R-18
51.4
FishRoPE (DINOv2-B + 2D RoPE)
DINOv2-B
52.5
FishRoPE (DINOv2-B + FishRoPE)
DINOv2-B
54.2
Table 1: 2D fisheye detection results. We report our WoodScape [email protected] and prior published numbers under their original protocols. Best in bold .
Method
Backbone
mIoU ↑
F2BEV [ 20 ]
R-18
53.39 †
FishBEV [ 1 ]
DINOv2-L
64.22 §
FishRoPE (R-18 + FishRoPE)
R-18
56.8
FishRoPE (DINOv2-B + 2D RoPE)
DINOv2-B
61.4
FishRoPE (DINOv2-B + FishRoPE)
DINOv2-B
65.1
Table 2: BEV semantic segmentation on SynWoodScapes. mIoU across foreground classes. Best in bold .
Position Encoding
mAP ↑
mIoU ↑
Learned absolute PE
51.8
59.2
Sinusoidal 2D PE
52.1
59.6
2D Axial RoPE [ 5 ]
52.5
61.4
FishRoPE ( θ only)
53.6
64.7
FishRoPE ( θ,ϕ )
54.2
65.1
Table 3: Position encoding ablation. Detection mAP (WoodScape) and BEV segmentation mIoU (SynWoodScapes). All use DINOv2-B backbone.
Backbone
Params
mAP ↑
mIoU ↑
Swin-T [ 15 ]
29M
52.9
59.8
DINOv2-S ∗ [ 16 ]
8.1M
53.4
62.7
DINOv2-B ∗ [ 16 ]
12.4M
54.2
65.1
Table 4: Backbone comparison. All use FishRoPE. mAP on WoodScape, mIoU on SynWoodScapes.
Figure 2: Qualitative results. FishRoPE (right) correctly handles peripheral image regions where baselines (left) fail due to fisheye distortion.
Rotary positional embeddings (RoPE) encode token positions by rotating attention features, supporting relative-position reasoning in language and multimodal models.