PQR3D: Progressive Query Refinement over Reference-Conditioned Temporal Windows for Multi-View 3D Object Detection

Hui Ye; Yudong Liu; Yiran Chen; Rajshekhar Sunderraman; Shihao Ji

cs.CVSep 26, 2026

PQR3D: Progressive Query Refinement over Reference-Conditioned Temporal Windows for Multi-View 3D Object Detection

Authors: Hui Ye, Yudong Liu, Yiran Chen, Rajshekhar Sunderraman, Shihao Ji

Organizations: Department of Computer Science, Georgia State University · Department of Electrical and Computer Engineering, Duke University · School of Computing, University of Connecticut

Abstract

Temporal context is essential for camera-only multi-view 3D object detection. Existing streaming detectors maintain and propagate query states from one frame to the next, requiring sequence-aware training and chronological inference. We propose PQR3D, which performs progressive query refinement within referenceconditioned temporal windows. This design enables random frame sampling and independent inference without persistent query memory. Within each window, PQR3D progressively transfers motion-aligned high-confidence queries from earlier timestamps toward the target frame. We further introduce masked selfattention to regulate interactions among regular, propagated, and denoising queries while keeping denoising supervision isolated from detection queries. In addition, a stage-decoupled anchor embedding injects position before self-attention and size, orientation, and velocity afterward, reducing interference from temporally inconsistent attributes. With a ViT-L backbone, PQR3D sets a new state of the art on the nuScenes test set, achieving 71.6 NDS and 64.9 mAP. Source code is available at https://github.com/huiyegit/PQR3D

Figures & tables

Figure 1: Comparison of temporal propagation strategies. (a) StreamPETR processes frames sequentially and passes query memory

M_{t}

from one frame to the next. (b) Sparse4D v2 performs sequential propagation by passing the instance bank

B_{t}

from one frame to the next. (c) PQR3D processes each reference-conditioned temporal window independently, without carrying memory across target samples. The dashed box shows one window,

W_{t_{j}}=\left\{F_{t_{j}-2},F_{t_{j}-1},F_{t_{j}}\right\}

, whose features are processed in temporal order to produce the 3D detection result

P_{t_{j}}

for the target frame. Here,

F_{t}

and

P_{t}

denote the multi-view image features and 3D detection result at time

t

, respectively.

Figure 2: Overall architecture of PQR3D. Within each temporal window, high-confidence queries are propagated from earlier frames toward the current frame, concatenated with the current queries $Q_{t}$ , and progressively refined by Transformer decoders. Each decoder contains $L$ layers (right), consisting of position embedding, masked self-attention, stage-decoupled anchor embedding, spatio-temporal sampling, and adaptive fusion.

Figure 3: Comparison of anchor embedding strategies. (a) Sparse4D v3 constructs the query and key by concatenating the anchor embedding

E

with the instance feature

F

, while deriving the value from

F

alone. (b) Our stage-decoupled anchor embedding injects the position embedding

E_{\mathrm{pos}}

before masked self-attention, while the size, yaw, and velocity embeddings are concatenated and injected afterward, before spatio-temporal sampling.

\oplus

denotes element-wise addition, and Cat denotes concatenation.

Table 3: Ablation studies on the nuScenes validation split using ResNet-50,

704\times 256

input resolution, and 24 training epochs. (a) Effect of each component. PQR: progressive query refinement; MSA: masked self-attention; SDAE: stage-decoupled anchor embedding; BF: BEV flip. (b) Comparison of temporal propagation strategies under the same detector and training settings. Vanilla uses no query propagation, whereas Sequential uses a StreamPETR-style strategy that carries cached queries from one frame to the next along the video sequence.

Figure 4: Qualitative comparison of SparseBEV (top) and PQR3D (bottom) on the nuScenes val set. Ground-truth boxes are shown in blue and predictions are class-colored (orange for car, crimson for bicycle, etc.). Red dashed circles mark regions of interest.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Speed–accuracy comparison of PQR3D with existing methods on the nuScenes validation set in terms of frames per second (FPS) and NDS.

Steps	NDS $\uparrow$	mAP $\uparrow$	Mem. (GB) $\downarrow$
0	55.5	45.3	8.1
1	56.2	46.1	8.3
2	56.6	46.5	8.7
3	56.8	46.4	9.3

Appendix

Table 4: Effect of the number of progressive refinement steps on detection accuracy and GPU memory usage.

Table 5: Ablation studies on propagated query count and anchor embedding design. (a) Number of propagated queries

k

. (b) Anchor embedding design. Pos: position-only embedding.

Figure 6: Qualitative comparison of SparseBEV (top) and PQR3D (bottom) on the nuScenes validation set. Ground-truth boxes are shown in blue, and predictions are class-colored. Red dashed circles indicate regions of interest.

Multi-View 3D Object Detection

Multi-View 3D Object Detection combines images from multiple cameras to locate and classify objects in three-dimensional space.

See all 25 papers

CAM3DNet: Comprehensively mining the multi-scale features for 3D Object Detection with Multi-View Cameras
Apr 18, 2026Mingxi Pang, Dingheng Wang, Zekun Li +4Multi-Scale Feature Fusion Multi-View 3D Object Detection
College of Intelligent Science and Technology, National University of Defense Technology, Changsha, 410073, China · Northwest Institute of Mechanical & Electrical Engineering, Xianyang, 712099, China. · School of Aerospace Engineering, Xi’an Jiaotong University, Xi’an, 710049, China
MAGneT-3D: Monocular and Domain-Generalizable Temporal 3D Detection
Aug 14, 2026Mohamed Kotb, Johannes Meier, Christoph Reich +33D Object Detection Monocular 3D Object Detection
TU Munich · ETH Zurich · MCML +4
Efficient Multi-View 3D Object Detection by Dynamic Token Selection and Fine-Tuning
Apr 15, 2026Danish Nazir, Antoine Hanna-Asaad, Lucas Görnhardt +3Multi-View 3D Object Detection 3D ViTs
Group Innovation, Volkswagen AG, 38440, Wolfsburg, Germany · Institute for Communications Technology, Technische Universität Braunschweig, 38106 Braunschweig, Germany

Apr 18, 2026J/K move · Enter open · S save

CAM3DNet: Comprehensively mining the multi-scale features for 3D Object Detection with Multi-View Cameras

Mingxi Pang, Dingheng Wang, Zekun Li, Zhenping Sun+3

College of Intelligent Science and Technology, National University of Defense Technology, Changsha, 410073, China · Northwest Institute of Mechanical & Electrical Engineering, Xianyang, 712099, China. · School of Aerospace Engineering, Xi’an Jiaotong University, Xi’an, 710049, China

Multi-Scale Feature Fusion Multi-View 3D Object Detection 3D Object Detection Deformable Attention

Open paper AskPDF arXiv

Query-based 3D object detection methods using multi-view images often struggle to efficiently leverage dynamic multi-scale information, e.g., the relationship between the object features and the geometric of the queries are not sufficiently learned, directly exploring the multi-scale spatiotemporal features will pay too many costs. To address these challenges, we propose CAM3DNet, a novel sparse query-based framework which combines three new modules, composite query (CQ), adaptive self-attention (ASA), and multi-scale hybrid sampling (MSHS). First, the core idea in the CQ module is a multi-scale projection strategy to transform 2D queries into 3D space. Second, the ASA module learns the interactions between the spatiotemporal multi-scale queries. Third, the MSHS module uses the deformable attention mechanism to sample multi-scale object information by considering multi-scales queries, pyramid feature maps, and 2D-camera prior knowledge. The entire model employs a backbone network and a feature pyramid network (FPN) as the encoder, then introduces a YOLOX and a DepthNet as a ROI_Head to produce CQ, and repeatedly utilizes ASA and MSHS as the decoder to gain detection features. Extensive experiments on the nuScenes, Waymo, and Argoverse benchmark datasets demonstrate the effectiveness of our CAM3DNet, and most existing camera-based 3D object detection methods are outperformed. Besides, we make comprehensive ablation studies to check the individual effect of CQ, ASA, and MSHS, as well as their cost of space and computation complexity.

Abstract

Multi-View 3D Object Detection

Explore similar work

CAM3DNet: Comprehensively mining the multi-scale features for 3D Object Detection with Multi-View Cameras

MAGneT-3D: Monocular and Domain-Generalizable Temporal 3D Detection

Efficient Multi-View 3D Object Detection by Dynamic Token Selection and Fine-Tuning