Multi-View Pedestrian Detection (MVPD) aims to detect pedestrians in the form of a bird's eye view map from multi-view images. Recent MVPD methods adopt a unified framework that projects 2D image features into a 3D world space and aggregates them into a single feature. Although they are effective, they struggle to generalize to unseen camera configurations during training due to two main issues. First, they are difficult to capture accurate visual geometry across views in unseen camera configurations. Second, they make detection models highly dependent on distortion patterns during training arising from their image feature projection. To address these, we leverage a visual geometric foundation model and propose MV2GF. This foundation model has exhibited strong generalization in capturing visual geometry across views and predicting accurate 3D attributes in diverse camera configurations. MV2GF fuses task-specific features with general-purpose geometric features extracted by the foundation model to effectively capture the visual geometry even in unseen camera configurations. Furthermore, MV2GF projects each pixel in the image features to an appropriate 3D location using 3D pointmaps predicted by the foundation model, preventing the detection model from depending on distortion patterns during training. Our experiments demonstrate the effectiveness of leveraging a visual geometric foundation model for MVPD and that MV2GF generalizes better than existing methods.
Figures & tables
Figure 1: (a) Previous MVPD methods. (b) Our MV2GF leverages a pretrained visual geometric foundation model. The red blocks are trainable, and the blue block is frozen.
Figure 2: (a) Overview of MV2GF. It consists of (b) TGF and (c) FPA. We set the red blocks to be trainable, and the gray blocks do not have trainable parameters.
Method
GMVD-D
MVPerception
Wildtrack
MODA
MODP
Prec.
Rec.
MODA
MODP
Prec.
Rec.
MODA
MODP
Prec.
Rec.
MVDet [ 17 ]
69.3
76.8
94.9
73.2
65.5
71.1
90.1
73.6
60.7
72.5
82.6
76.9
SHOT [ 39 ]
71.5
77.9
93.7
76.7
69.4
73.1
91.1
76.8
70.8
73.1
82.3
90.2
MVDeTr [ 16 ]
73.8
81.7
96.1
76.9
71.3
78.0
92.8
77.3
72.4
78.1
85.6
87.1
3DROM [ 34 ]
74.5
77.8
93.7
80.0
74.1
75.6
91.1
82.1
76.2
75.7
85.2
92.1
BoosterSHOT [ 20 ]
74.9
80.2
96.2
77.8
74.5
76.4
87.4
87.1
78.6
79.0
87.4
91.8
Table 1: Comparison with previous methods under settings where camera configurations during testing are different from those during training. All models are trained on GMVD-D training split.
Method
GMVD-S
MVPerception
Wildtrack
MODA
MODP
Prec.
Rec.
MODA
MODP
Prec.
Rec.
MODA
MODP
Prec.
Rec.
MVDet [ 17 ]
82.8
80.4
97.1
85.3
90.0
81.2
94.3
95.8
88.2
75.7
94.7
93.6
SHOT [ 39 ]
84.5
79.6
97.5
86.7
92.3
85.5
95.2
97.1
90.2
76.5
96.1
94.0
MVDeTr [ 16 ]
86.9
84.6
97.4
89.3
93.0
90.2
96.5
96.6
91.5
82.1
97.4
94.0
3DROM [ 34 ]
88.1
84.7
97.0
90.5
94.1
85.5
97.4
96.7
93.5
75.9
97.2
96.2
BoosterSHOT [ 20 ]
88.7
84.9
97.0
91.5
94.5
88.5
98.3
96.1
92.8
84.9
97.5
95.3
Table 2: Comparison with previous methods under settings where camera configurations during testing are identical to those during training. All models are trained on the training split of the same dataset used for testing.
Table 3: (a) Effect of TGF and FPA. (b) Effect of using features from ResNet and DA3 in TGF. (c) Effect of using features from multiple transformer blocks in DA3. (d) Effect of θz in FPA. (e) Effect of using features from a visual geometric foundation model. (f) Effect of the visual geometric foundation model choice. Default settings in MV2GF are marked in gray . All models are trained on GMVD-D training split and tested on its testing split.
MSMVD
Ours
N
MODA
MODP
Prec.
Rec.
MODA
MODP
Prec.
Rec.
6
82.7
81.1
95.2
87.1
85.9
82.0
96.8
88.9
5
79.9
79.0
93.0
86.3
84.1
80.5
96.8
87.0
4
70.5
78.7
88.6
80.9
80.0
80.3
95.5
84.0
3
61.4
72.6
85.0
74.5
73.8
78.0
93.7
79.1
2
34.0
63.0
76.0
49.8
57.6
73.3
86.3
67.6
Table 4: Effect of the number of input views N for MSMVD [ 50 ] and our MV2GF. Both models are trained on GMVD-D training split and tested on data comprising 6 views in GMVD-D testing split.
FPS
MVDet [ 17 ]
5.2
SHOT [ 39 ]
4.8
MVDeTr [ 16 ]
4.6
3DROM [ 34 ]
4.6
BoosterSHOT [ 20 ]
4.7
OmniOcc [ 3 ]
4.6
Table 5: Comparison of inference speed. These inference speeds are measured on data comprising 6 views in GMVD-D testing split using an Nvidia 80GB A100 GPU.
Pooling MODA MODP Prec. Rec. w/ loss weighting 84.2 83.1 96.6 87.0 w/o loss weighting 84.8 83.1 96.8 87.7
(e)
(f)
Table 6: (a) Effect of the offset prediction. (b) Effect of auxiliary loss Lviewn . (c) Effect of ResNet size in TGF. (d) Effect of the pooling choice in FPA. (e) Effect of using features from a trainable ResNet. (f) Effect of loss weighting. Default settings in MV2GF are marked in gray . All models are trained on GMVD-D training split and tested on its testing split.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
MODA
MODP
Prec.
Rec.
MSMVD [ 50 ]
91.1
86.6
98.0
93.0
Ours w/o MSMVD’s multi-scale BEV design
91.3
86.1
97.2
93.9
Ours w/ MSMVD’s multi-scale BEV design
91.8
87.2
98.1
94.0
Appendix
Table 7: Effect of MSMVD’s multi-scale BEV design on GMVD-S. All models are trained on GMVD-S training split and tested on GMVD-S testing split. Camera configurations during training are identical to those during testing. Default settings in MV2GF are marked in gray .
MODA
MODP
Prec.
Rec.
MVDet [ 17 ]
17.0
65.8
60.5
48.8
SHOT [ 39 ]
53.6
72.0
75.2
79.8
MVDeTr [ 16 ]
50.2
69.1
74.0
77.3
3DROM [ 34 ]
67.5
65.6
94.5
71.7
MVFP [ 2 ]
82.6
76.2
89.6
93.4
MSMVD [ 50 ]
82.0
78.0
88.7
94.0
Appendix
Table 8: Comparison with previous methods under the setting where models are trained on MultiviewX and tested on Wildtrack.
Figure 3: Visual comparison of detection results ( i.e . , predicted BEV maps) between our MV2GF and MSMVD [ 50 ] on GMVD-D. The green points represent detected pedestrians, the blue circles represent false positives, and the red circles represent false negatives.
Figure 4: Visual comparison of detection results ( i.e . , predicted BEV maps) between our MV2GF and MSMVD [ 50 ] on MVPerception. The green points represent detected pedestrians, the blue circles represent false positives, and the red circles represent false negatives.
Figure 5: Visual comparison of detection results ( i.e . , predicted BEV maps) between our MV2GF and MSMVD [ 50 ] on Wildtrack. The green points represent detected pedestrians, the blue circles represent false positives, and the red circles represent false negatives.
Figure 6: Visualization of 3D pointmaps predicted by DA3 [ 28 ] on GMVD.
Figure 7: Visualization of 3D pointmaps predicted by DA3 [ 28 ] on MVPerception.
Figure 8: Visualization of 3D pointmaps predicted by DA3 [ 28 ] on Wildtrack.
Figure 9: Visualization of (a) the ground truth BEV map and BEV features of (b) our MV2GF and (c) MVFP [ 2 ] on GMVD-D.
The core challenge in multi-view pedestrian detection (MVPD) lies in effective aggregation of visual features from different viewpoints for robust occlusion reasoning. Recent approaches have addressed this by first projecting image-view features onto a Bird's Eye View (BEV) map, where ground localization is then performed. Despite impressive performance, the perspective transformation induces severe distortion, causing spatial structure break and degrading the quality of object feature extraction. The blurred and ambiguous features hinder accurate BEV point localization, especially in densely populated regions. Moreover, the strong mutual relationship between the BEV ground point and image bounding boxes is not capitalized on. Although multi-view consistency of 2D detections can serve as a powerful constraint in BEV space, these detections are commonly treated as auxiliary signals rather than being jointly optimized with the primary task.In this work, we propose MVDGC, a unified framework that jointly estimates pedestrian locations on the BEV plane and 2D bounding boxes in image views. MVDGC employs a sparse set of 3D cylindrical queries that embraces geometric context across both BEV and image views, enforcing dual spatial constraints for precise localization. Specifically, the geometric constraints is established by modeling each pedestrian as a vertical cylinder whose center lies on the BEV plane and whose projection casts a rectangular box in the image views. These queries function as shape anchors that directly extract 2D features from the intact image-view features using camera projection, eliminating projection-induced distortions. The 3D cylindrical query enables the unification of BEV and ImV localization into a single task: 3D cylinder position and shape refinement. Code is available at: https://github.com/UARK-AICV/MVDGC
Thinh Phan, Hao Vo, Khoa Vo +3
AICV Lab, EECS Department, University of Arkansas · University of Information Technology · Posts and Telecommunications Institute of Technology (PTIT), Vietnam
Computing accurate geometry from multi-view images is a fundamental problem in computer vision. Recent feed-forward (FF) models jointly estimate 3D geometry and camera parameters, but they typically suffer from geometry distortion caused by reconstruction ambiguity, even when ground-truth camera parameters are supplied. In this paper, we study the multi-view stereo (MVS) problem with known camera parameters and propose a novel approach that bridges conventional MVS and FF methods. Rather than casting MVS as a sequence-to-one mapping that predicts depth only for a single reference view, we reformulate it as a sequence-to-sequence task, akin to FF models, that jointly predicts geometry for all input views. We introduce a global transformer-based architecture with two components that explicitly exploit camera-induced priors: ray-map embeddings that inject camera parameters into image patch tokens, making the transformer camera-aware, and a unified global cost volume that replaces conventional per-view cost volumes to jointly capture 3D structure across all views. Extensive experiments on multiple public benchmarks show our approach achieves state-of-the-art performance, surpassing both MVS and FF reconstruction baselines.
Multi-view anomaly detection (MvAD) detects defects by exploiting complementary observations from multiple camera viewpoints. The central challenge is to fuse views with sufficient geometric awareness while remaining scalable to multi-class industrial settings. Existing methods typically fall into two extremes: voxel-based fusion provides explicit geometric alignment but requires costly 3D construction and class-specific assumptions, whereas lightweight patch-based fusion is efficient but relies on discrete candidate matching and lacks continuous cross-view correspondence. In this paper, we propose GeoMAD, a unified multi-view, multi-class AD framework that addresses both geometric correspondence deficiency and distributional inconsistency. Our Cross-view Deformable Fusion Module (CDFM) learns content-adaptive, view-pair-specific sampling offsets directly on 2D feature maps and arranges them across a multi-scale window pyramid with image-global reference sampling, enabling hierarchical cross-view correspondence without camera calibration, voxel construction, or class-specific 3D supervision. We further introduce Distributional View Alignment (DVA), a self-supervised cross-view regularization loss that aligns each view's bottleneck distribution against a per-instance view-centric target, enforcing global consistency without pixel-level correspondence. Together, CDFM and DVA bridge local geometric correspondence and global distributional consistency, providing geometry-aware and distribution-consistent fusion while preserving the efficiency of 2D feature-space learning. Extensive experiments on Real-IAD and MANTA-Tiny show that GeoMAD achieves strong detection and localization performance in unified MvAD.
Shang-Fu Chen, Jhih-Ciang Wu, Kuan-Chuan Peng +2
National Taiwan University Taipei, Taiwan · National Taiwan Normal University Taipei, Taiwan · Mitsubishi Electric Research Laboratories (MERL) Cambridge, MA, USA +3