PC-CrossDiff: Point-Cluster Dual-Level Cross-Modal Differential Attention for Unified 3D Referring and Segmentation

Wenbin Tan; Jiawen Lin; Fangyong Wang; Yuan Xie; Yong Xie; Yachao Zhang; Yanyun Qu

cs.CVMar 18, 2026

PC-CrossDiff: Point-Cluster Dual-Level Cross-Modal Differential Attention for Unified 3D Referring and Segmentation

Authors: Wenbin Tan, Jiawen Lin, Fangyong Wang, Yuan Xie, Yong Xie, Yachao Zhang, Yanyun Qu

Organizations: Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, School of Informatics, Xiamen University, Xiamen, China · School of Big Data, Tongren University, Tongren, China · Hanjiang National Laboratory, Wuhan, China · School of Computer Science and Technology, East China Normal University, Shanghai, China ·

Abstract

3D Visual Grounding (3DVG) aims to localize the referent of natural language referring expressions through two core tasks: Referring Expression Comprehension (3DREC) and Segmentation (3DRES). While existing methods achieve high accuracy in simple, single-object scenes, they suffer from severe performance degradation in complex, multi-object scenes that are common in real-world settings, hindering practical deployment. Existing methods face two key challenges in complex, multi-object scenes: inadequate parsing of implicit localization cues critical for disambiguating visually similar objects, and ineffective suppression of dynamic spatial interference from co-occurring objects, resulting in degraded grounding accuracy. To address these challenges, we propose PC-CrossDiff, a unified dual-task framework with a dual-level cross-modal differential attention architecture for 3DREC and 3DRES. Specifically, the framework introduces: (i) Point-Level Differential Attention (PLDA) modules that apply bidirectional differential attention between text and point clouds, adaptively extracting implicit localization cues via learnable weights to improve discriminative representation; (ii) Cluster-Level Differential Attention (CLDA) modules that establish a hierarchical attention mechanism to adaptively enhance localization-relevant spatial relationships while suppressing ambiguous or irrelevant spatial relations through a localization-aware differential attention block. Our method achieves state-of-the-art performance on the ScanRefer, NR3D, and SR3D benchmarks. Notably, on the Implicit subsets of ScanRefer, it improves the [email protected] score by +10.16% for the 3DREC task, highlighting its strong ability to parse implicit spatial cues.

Figures & tables

Figure 1: PC-CrossDiff vs. SOTA on 3DREC and 3DRES. ScanRefer (IoU@0.50), NR3D/SR3D (IoU@0.25). — Figure 1: PC-CrossDiff vs. SOTA on 3DREC and 3DRES. ScanRefer ([email protected]), NR3D/SR3D ([email protected]).

Figure 2: The framework of PC-CrossDiff. Text and visual features are input into the PLDA module to extract implicit localization cues and enhance point-level visual features. Simultaneously, the coordinates $F_{\mathrm{xyz}}$ are fed into the CLDA module for Spatial Relation Modeling (SRM), modeling cluster-level relations. These features are enhanced by LDA and integrated into the aligned features $F^{\prime}_{\text{v}}$ and $F^{\prime}_{\text{t}}$ , yielding $\tilde{F}_{\text{v}}^{\prime}$ and $\tilde{F}_{\text{t}}^{\prime}$ . The enhanced features are decoded by 3DREC and 3DRES heads to generate predictions, with $\mathcal{L}_{\text{DGTL}}$ facilitating task synergy.

Figure 3: LDA block. With $\mathit{F^{\prime}}_{\mathrm{v}}$ as query, DiffAtten ( Eq. 6 ) extracts relevant relations from $\mathit{F^{\prime}}_{\mathrm{oRel}}$ while suppressing irrelevant ones. Features then pass self-attention to yield $\mathit{\tilde{F}}_{\mathrm{v}}^{\prime}$ .

Method	Venue	Overall		mIoU	FPS
		0.25	0.50
TGNN	AAAI’21	37.50	31.40	27.80	2.85
X-RefSeg3D	AAAI’24	40.33	33.77	29.94	2.03
3D-STMN	AAAI’24	54.60	39.80	39.50	4.14
SegPoint	ECCV’24	-	-	41.70	-
MCLN	ECCV’24	58.70	50.70	44.72	4.81

Table 1: Comparison results of 3DRES on the ScanRefer dataset (inference: batchsize=1, on an NVIDIA 3090 GPU).

3DRES
Method	Multiple		Implicit
	0.25	0.50	0.25	0.50
X-RefSeg3D	28.16	23.61	43.71	37.33
3D-STMN	46.20	29.20	57.07	40.32
RefMask3D	48.09	40.77	49.30	44.71
MCLN	53.28	45.88	59.16	51.99

Table 2: Joint Evaluation of 3DRES and 3DREC across Multiple and Implicit Subsets in the ScanRefer Dataset.

Method	Venue	Multiple		Overall
		0.25	0.50	0.25	0.50
Two-stage Model
MVT	CVPR’22	31.92	25.26	40.80	33.26
3D-SPS	CVPR’22	40.32	29.82	48.82	36.98
BUTD-DETR	ECCV’22	44.73	33.97	50.42	38.60
ViL3DRel	NIPS’22	40.30	30.71	47.94	37.73

Table 3: Comparison results of single-stage and two-stage 3DREC on the ScanRefer dataset (*: Reproduction).

Method	Venue	SR3D	NR3D
3D-SPS	CVPR’22	62.60	51.50
MVT	CVPR’22	64.50	55.10
BUTD-DETR	ECCV’22	65.60	49.10
LAR	NIPS’22	59.60	48.90
EDA	CVPR’23	68.10	52.10
VPP-Net	CVPR’24	68.70	56.90

Table 4: 3DREC performance on SR3D and NR3D ([email protected]).

Component			3DREC		3DRES
$\bm{\mathcal{L_{\text{DGTL}}}}$	PLDA	CLDA	0.25	0.50	0.25	0.50	mIoU
w/o	w/o	w/o	56.76	44.86	58.43	49.66	43.85
$\checkmark$	w/o	w/o	56.95	45.50	58.49	50.86	44.78
$\checkmark$	$\checkmark$	w/o	57.56	46.20	59.19	50.99	45.12
$\checkmark$	w/o	$\checkmark$	57.69	46.18	59.61	51.88	45.68
$\checkmark$	$\checkmark$	$\checkmark$	58.47	47.89	60.41	52.52	46.39

Table 5: Ablation study of

\mathcal{L_{\text{DGTL}}}

, PLDA, and CLDA on 3DREC/3DRES (

\checkmark

: enabled; w/o: disabled).

CLDA		3DREC		3DRES
$N_{\mathrm{clust}}$	$S_{\mathrm{clust}}$	0.25	0.50	0.25	0.50	mIoU
256	4	56.11	46.37	57.83	50.88	44.66
128	8	58.15	47.09	60.21	52.40	46.16
64	16	58.47	47.89	60.41	52.52	46.39
32	32	57.36	47.01	59.25	51.53	45.54

Table 6: Ablation study of cluster configurations (

N_{\mathrm{clust}}

and

S_{\mathrm{clust}}

) on 3DREC and 3DRES tasks.

Component	3DREC		3DRES
	0.25	0.50	0.25	0.50	mIoU
$F^{(4)}_{\mathrm{v}}$	56.94	45.98	58.87	50.97	44.94
$\mathcal{K}^{\mathrm{t}2\mathrm{v}}_{\lambda}$	58.06	46.92	60.25	51.84	46.13
$F_{\mathrm{t}}$	57.93	46.71	60.02	52.42	46.27
$\mathcal{K}^{\mathrm{v}2\mathrm{t}}_{\lambda}$	58.47	47.89	60.41	52.52	46.39

Table 7: Comparison of localization accuracy when integrating different feature configurations from PLDA into CLDA.

Component	3DREC		3DRES
	0.25	0.50	0.25	0.50	mIoU
$\star$	56.64	45.81	58.54	50.63	44.85
PLDA+CLDA	58.47	47.89	60.41	52.52	46.39

Table 8: Performance comparison of our dual-level differential attention and standard cross-attention (

\star

: standard Transformer-based ( Vaswani et al. 2017 ) ).

Figure 4: Comparison of PC-CrossDiff (ours) and MCLN (baseline) for 3DREC and 3DRES with implicit cues in complex multi-object scenes. GT boxes are yellow.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

F^{\prime}_{\mathrm{oRel}}=\mathrm{DGCNN}(F_{\mathrm{iRel}},\mathbf{O})\in\mathbb{R}^{B\times N_{\mathrm{clust}}\times d_{\mathrm{mod}}}.

Appendix

Algorithm 1 The Working Process of CLDA.

Method	3DREC		3DRES
	0.25	0.50	0.25	0.50	mIoU
MCLN	68.43	57.30	64.70	57.88	49.84
Ours	70.95	61.27	68.70	62.04	53.43

Appendix

Table 9: Performance comparison of 3DREC and 3DRES on the SR3D dataset.

Method	3DREC		3DRES
	0.25	0.50	0.25	0.50	mIoU
MCLN †	53.75	45.82	51.79	46.58	40.47
Ours	59.91	50.48	57.56	52.64	46.00

Appendix

Table 10: Performance comparison of joint 3DREC and 3DRES tasks on the NR3D dataset ( † reproduced results; 200 epochs).

PLDA / CLDA		3DREC		3DRES
		0.25	0.50	0.25	0.50
w/o	w/o	51.96	40.76	53.28	45.88
$\checkmark$	w/o	52.35	41.51	53.84	46.67
w/o	$\checkmark$	52.74	41.50	54.57	47.56
$\checkmark$	$\checkmark$	53.59	43.12	55.48	47.98

Appendix

Table 11: Ablation study of PLDA and CLDA on ScanRefer-Multiple for 3DREC and 3DRES.

\checkmark

: included; w/o : no CLDA.

CLDA		3DREC		3DRES
$N_{\mathrm{clust}}$	$S_{\mathrm{clust}}$	0.25	0.50	0.25	0.50	mIoU
/	/	56.76	44.86	58.43	49.66	43.85
256	4	56.11	46.37	57.83	50.88	44.66
128	8	58.15	47.09	60.21	52.40	46.16
64	16	58.47	47.89	60.41	52.52	46.39
32	32	57.36	47.01	59.25	51.53	45.54

Appendix

Table 12: Ablation study of cluster configurations (

N_{\mathrm{clust}}

and

S_{\mathrm{clust}}

) on 3DREC and 3DRES tasks. / denotes no CLDA.

3D Referring Segmentation

See all 13 papers

PruneGround: Plug-and-play Spatial Pruning for 3D Visual Grounding
Jun 30, 2026Duc Cao Dinh, Khai Le-Duc, Florent Draye +43D Spatial Reasoning Visually Grounded Reasoning
Knovel Engineering Lab · University of Toronto · Vector Institute +3
Hierarchical Collaborative Fusion for 3D Instance-aware Referring Expression Segmentation
Mar 6, 2026Keshen Zhou, Runnan Chen, Mingming Gong +13D Referring Segmentation Referring Expression Segmentation
The University of Sydney · The University of Melbourne · Mohamed bin Zayed University of Artificial Intelligence
3D-DRES: Detailed 3D Referring Expression Segmentation
Mar 3, 2026Qi Chen, Changli Wu, Jiayi Ji +23D Referring Segmentation Referring Expression Segmentation
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China · Shanghai Innovation Institute · National University of Singapore

Jun 30, 2026J/K move · Enter open · S save

PruneGround: Plug-and-play Spatial Pruning for 3D Visual Grounding

Duc Cao Dinh, Khai Le-Duc, Florent Draye, Chris Ngo+3

Knovel Engineering Lab · University of Toronto · Vector Institute+3

3D Spatial Reasoning Visually Grounded Reasoning 3D VLMs 3D Visual Grounding

Open paper AskPDF arXiv

3D Visual Grounding (3DVG) aims to localize target objects in 3D scenes given natural language descriptions. Existing approaches typically perform reasoning over the entire scene, leading to ambiguous predictions and high computational cost, especially in cluttered environments. We observe that many referential expressions rely on local spatial context and often correspond to restricted spatial regions rather than the full scene. Motivated by this insight, we propose PruneGround, an effective plug-and-play framework for 3DVG built upon three key components. First, we introduce Language-Guided Spatial Pruning (LGSP), which leverages a frozen Vision Language Model (VLM) to identify language-relevant regions, thereby reducing spatial computation and grounding candidates in the narrower search space. Second, we propose MultiView-Conditioned Description Reformulation (MCDR), which decomposes complex expressions into simplified target--anchor relations and augments missing spatial cues through multi-view reasoning. Finally, we propose LLM-Grounder, which repurposes a detection-pretrained spatial LLM into a language-conditioned grounding model by aligning point cloud and linguistic representations within the pruned region. Extensive experiments on the three most popular point cloud benchmarks demonstrate that our method achieves state-of-the-art results on all three ScanRefer settings and on 9 out of 10 Nr3D/Sr3D settings. Code and models are publicly available: https://github.com/leduckhai/PruneGround

Abstract

3D Referring Segmentation

Explore similar work

PruneGround: Plug-and-play Spatial Pruning for 3D Visual Grounding

Hierarchical Collaborative Fusion for 3D Instance-aware Referring Expression Segmentation

3D-DRES: Detailed 3D Referring Expression Segmentation