A3R: Agentic Affordance Reasoning via Cross-Dimensional Evidence in 3D Gaussian Scenes

Di Li; Jie Feng; Guanbin Li; Ronghua Shang; Yuhui Zheng; Weisheng Dong; Guangming Shi

cs.CVApr 2, 2026

A3R: Agentic Affordance Reasoning via Cross-Dimensional Evidence in 3D Gaussian Scenes

Authors: Di Li, Jie Feng, Guanbin Li, Ronghua Shang, Yuhui Zheng, Weisheng Dong, Guangming Shi

Organizations: Xidian University, China · Sun Yat-sen University, China · Qinghai Normal University, China

Abstract

Affordance reasoning in 3D Gaussian scenes aims to identify the region that supports the action specified by a given text instruction in complex environments. Existing methods typically cast this problem as one-shot prediction from static scene observations, assuming sufficient evidence is already available for reasoning. However, in complex 3D scenes, many failure cases arise not from weak prediction capacity, but from incomplete task-relevant evidence under fixed observations. To address this limitation, we reformulate fine-grained affordance reasoning as a sequential evidence acquisition process, where ambiguity is progressively reduced through complementary 3D geometric and 2D semantic evidence. Building on this formulation, we propose A3R, an agentic affordance reasoning framework that enables an MLLM-based policy to iteratively select evidence acquisition actions and update the affordance belief through cross-dimensional evidence acquisition. To optimize such sequential decision making, we further introduce a GRPO-based policy learning strategy that improves evidence acquisition efficiency and reasoning accuracy. Extensive experiments on scene-level benchmarks show that A3R consistently surpasses static one-shot baselines, demonstrating the advantage of agentic cross-dimensional evidence acquisition for fine-grained affordance reasoning in complex 3D Gaussian scenes.

Figures & tables

Figure 1. Affordance reasoning in complex 3DGS scenes is often limited by insufficient task-relevant evidence under static observation. Our framework addresses this issue through agentic cross-dimensional evidence acquisition, which progressively gathers complementary 3D geometric and 2D semantic evidence to reduce ambiguity and produce accurate fine-grained predictions. Enjoying

Method	Venue	Scene Rep.	Inference Mode	Cross-Dimensional Evidence
3DAffordanceNet ( Deng et al., 2021 )	CVPR’21	Point cloud	Static one-shot	None
IAGNet ( Yang et al., 2023 )	ICCV’23	Point cloud	Static one-shot	Static Fusion
PointRefer ( Li et al., 2024 )	CVPR’24	Point cloud	Static one-shot	None
AffordSplatNet ( Wei et al., 2025 )	MM’25	3DGS	Static one-shot	Static Fusion
SeqAffordSplat ( Li et al., 2025b )	arXiv’25	3DGS	Static one-shot	Static Fusion
AffordBot ( Wang et al., 2025b )	NeurIPS’25	Point cloud	Active viewpoint selection	View-mediated

Table 1. Comparison with representative affordance reasoning methods. Our method differs from prior work by performing agentic cross-dimensional evidence acquisition in 3DGS scenes with both 2D and 3D tools.

Figure 2. Overview of the proposed framework. The agent maintains a scene-grounded decision state composed of the current workspace, affordance belief, and action–evidence history. Conditioned on this state, the policy selects 2D or 3D evidence operators whose outputs are projected back to the scene support to update the state, progressively reducing ambiguity for fine-grained affordance reasoning in 3DGS scenes. Overview of the proposed framework for agentic cross-dimensional evidence acquisition in 3DGS scenes.

Setting	Method	mIoU $\uparrow$	AUC $\uparrow$	SIM $\uparrow$
Seen	IAGNet ( Yang et al., 2023 )	9.70	79.45	0.168
	PointRefer ( Li et al., 2024 )	17.61	85.90	0.260
	AffordSplatNet ( Wei et al., 2025 )	21.04	83.37	0.283
	Ours (training-free)	26.81	86.11	0.330
	Ours (RL)	28.28	88.51	0.355
Unseen	IAGNet ( Yang et al., 2023 )	10.48	61.05	0.176

Table 2. Main results on SeqAffordSplat.

Setting	Method	mIoU $\uparrow$	AUC $\uparrow$	SIM $\uparrow$
Seen	IAGNet ( Yang et al., 2023 )	14.63	56.67	0.350
	PointRefer ( Li et al., 2024 )	18.40	78.50	0.430
	AffordSplatNet ( Wei et al., 2025 )	30.25	83.85	0.440
	Ours (training-free)	34.88	81.86	0.462
	Ours (RL)	36.30	83.90	0.475
Unseen	IAGNet ( Yang et al., 2023 )	4.70	40.77	0.240

Table 3. Main results on 3DAffordSplat.

Figure 3. Qualitative comparison with static baselines. Our method produces more accurate and spatially precise affordance predictions, especially in ambiguity-heavy scenes that require fine-grained localization and distractor suppression. Qualitative comparison with static baselines for affordance reasoning.

Figure 4. Visualization of evidence-acquisition trajectories. The agent adaptively selects 2D semantic and 3D geometric operators based on the current belief, progressively refining the affordance prediction through coarse-to-fine cross-dimensional evidence acquisition. Visualization of evidence-acquisition trajectories for agentic cross-dimensional reasoning.

Strategy	mIoU $\uparrow$	AUC $\uparrow$	SIM $\uparrow$	Avg. Steps $\downarrow$
SAM2D only	18.40	72.20	0.230	1
Part only	21.04	83.37	0.283	1
Static fusion	19.10	82.89	0.267	1
Fixed Seq. A	21.97	82.01	0.302	3
Fixed Seq. B	18.96	85.39	0.212	2
Fixed Seq. C	21.80	84.21	0.248	3

Table 4. Comparison of evidence acquisition strategies. All methods use the same evidence operators and the same backbone.

No.	SAM2D	Part	Inst	Zoom	mIoU $\uparrow$	AUC $\uparrow$	SIM $\uparrow$
1		✓			21.04	83.37	0.28
2	✓				18.40	72.20	0.23
3	✓	✓			23.04	84.55	0.29
4	✓	✓	✓		26.11	86.01	0.31
5	✓	✓	✓	✓	26.81	86.11	0.33

Table 5. Ablation study of the proposed evidence components.

Agent Backbone	mIoU $\uparrow$	AUC $\uparrow$	SIM $\uparrow$	Avg. Steps $\downarrow$
Qwen3-VL-2B	19.98	75.56	0.24	4.96
Qwen3-VL-4B	22.63	85.00	0.28	3.92
Qwen3-VL-8B	26.81	86.11	0.33	2.66

Table 6. Effect of agent backbone choice.

3D Scene Understanding

3D scene understanding builds geometric and semantic representations of environments for spatial reasoning, perception, and interaction.

See all 295 papers

ThinkAfford: Affordance-Centric Reasoning for Fine-Grained 3D Grounding in Cluttered Scenes
Aug 11, 2026Xinrui Lin, Sha Zhang, Shumin Wang +33D Affordance Grounding 3D Scene Understanding
University of Science and Technology of China, Hefei, China · The Chinese University of Hong Kong, Hong Kong SAR, China
MAG-3D: Multi-Agent Grounded Reasoning for 3D Understanding
Apr 10, 2026Henry Zheng, Chenyue Fang, Rui Huang +33D Spatial Reasoning 3D Scene Understanding
Tsinghua University · Pico, ByteDance
Grounding by Remembering: Cross-Scene and In-Scene Memory for 3D Functional Affordances
May 12, 2026Qirui Wang, Jingyi He, Yining Pan +23D Affordance Grounding Visually Grounded Reasoning
TUM · A*STAR

Aug 11, 2026J/K move · Enter open · S save

ThinkAfford: Affordance-Centric Reasoning for Fine-Grained 3D Grounding in Cluttered Scenes

Xinrui Lin, Sha Zhang, Shumin Wang, Zenghuan Zhu+2

University of Science and Technology of China, Hefei, China · The Chinese University of Hong Kong, Hong Kong SAR, China

3D Affordance Grounding 3D Scene Understanding 3D VLMs 3D Visual Grounding

Open paper AskPDF arXiv

Task-driven 3D affordance grounding aims to localize the functional region in a cluttered 3D scene that enables an action specified by a natural-language instruction. Existing methods either predict 3D masks directly or construct them by selecting and fusing intermediate 2D/3D regions. However, they remain vulnerable to two intertwined failure modes: the predicted or selected regions may miss the target interaction area or have unsuitable granularity, while language grounding may confuse visually similar alternatives under relational instructions. To this end, we introduce ThinkAfford, which decouples high-recall affordance proposal generation from instruction-grounded reasoning. Specifically, the Affordance Proposal Generation module first uses learnable affordance prompts and multi-level visual features to predict interaction-conditioned heatmaps, extracting a variable number of fine-grained proposals without parsed object or part names as segmentation prompts. Visual-Prompted Affordance Reasoning then reasons over labeled proposal overlays using the full instruction, returning identifiers in a structured "think-then-answer" response. Moreover, Group Relative Policy Optimization uses proposal-level rewards from lifted 3D overlap to align VPAR selection with final 3D grounding. On the SceneFun3D validation split, ThinkAfford achieves 10.69% AP50 and 25.46% AP25 under the official evaluator, outperforming comparable 3D open-vocabulary and vision-language-model-based 2D-to-3D baselines. Module-level diagnostics further show that APG attains 77.5% recall at 25% intersection-over-union, while GRPO-trained VPAR achieves 72.1% selection accuracy on APG-covered queries, compared with 63.4% under supervised fine-tuning.

Abstract

3D Scene Understanding

Explore similar work

ThinkAfford: Affordance-Centric Reasoning for Fine-Grained 3D Grounding in Cluttered Scenes

MAG-3D: Multi-Agent Grounded Reasoning for 3D Understanding

Grounding by Remembering: Cross-Scene and In-Scene Memory for 3D Functional Affordances