Grounding Generative Verification in 3D Content

Shuanglin Li; Jiaying Lin; Shuquan Ye

cs.CVOct 15, 2026

Grounding Generative Verification in 3D Content

Authors: Shuanglin Li, Jiaying Lin, Shuquan Ye

Organizations: Southeast University · Shenzhen Loop Area Institute · The Hong Kong University of Science and Technology

Abstract

Recent advances in multimodal large language models (MLLMs) and 3D generation have expanded 3D understanding and synthesis. However, beyond visual fidelity, task-specific correctness grounded in 3D evidence remains a major challenge. We argue that building reliable 3D systems requires not only stronger generation and reasoning, but also verification feedback that identifies unmet requirements and explains their violations, allowing systems to reflect on and refine their outputs. Such feedback must integrate multi-view observations and distinguish genuine inconsistencies from projection-dependent appearances. We make three contributions. First, we introduce 3DVerifierBench, a benchmark spanning 10 tasks that evaluates decisions, rejection explanations, and responses to controlled changes in specifications and candidates. A human study reveals a 21.6 percentage-point gap between humans and the strongest tested model, GPT-6 Astra, while explanation-aware analysis exposes weaknesses in rejection reliability. Second, we analyze verification across fine-grained factual categories, exposing errors in values and target binding. We develop a scalable pipeline combining multi-view fact extraction, controlled 3D editing, and native annotations to produce 14,126 training examples with contradiction explanations and linked counterfactuals. Third, we examine how verification could support 3D generation through candidate selection and recursive refinement. In human-labeled sixteen-candidate pools, oracle selection improves complete factual success from 26.7% to 90.0%. In a separate eight-input study, human-guided recursive refinement satisfies all seven requirements for every input, compared with three under parallel Oracle@8. Together, our benchmark, analysis, and data pipeline establish a foundation for models to reflect on 3D content through reliable verification feedback.

Figures & tables

Figure 1: Even state-of-the-art MLLMs, e.g. , GPT-6 Astra, still struggle with 3D content verification. Top: Astra mistakenly accepts an unnatural rightward collapse and an identity-changing face edit. Bottom: We found that Astra is still far behind human performance across all tasks in our 3DVerifierBench. Views and text are abbreviated for display.

Panel of Figure 2 — Figure 2: Evaluation scope relative to prior work. Prior work ( Zhang et al., 2025a ; Zhang et al., 2025b ; Zhang et al., 2026b ; Duggal et al., 2025 ) emphasizes preference, hierarchical quality, general visual verification, or asset-level consistency and artifact localization. Our 3DVerifierBench targets complementary verifier-side questions through explicit requirements, controlled counterfactuals, heterogeneous 3D evidence, and verification-guided refinement.

Figure 3: Overview of 3DVerifierBench. Each task presents task-specific 3D evidence together with a verification requirement, reference answer, and explanation. Evidence spans multi-view objects, marked parts and regions, grasp poses, identity comparisons, photo-to-geometry comparisons, and motion sequences. The table summarizes the number of examples and input evidence for each task, totaling 2,581 examples. Mat., Ground., Aff. Reg., and FineGeo. denote Material, Part Grounding, Affordance Region, and Fine-grained Geometry, respectively. Views and text are abbreviated.

Figure 4: Our pipeline. Reviewed evidence establishes positive anchors. Spec-CF changes the requirement under fixed evidence, whereas Cand-CF changes the candidate under the original requirement. Native annotations and task-specific evidence complement these controlled mutations. Automated checks, model-assisted review, and human review together produce the benchmark.

Model / Evaluator	Object	Part	Mat.	Ground.	Relation	Aff. Reg.	Grasp	Identity	FineGeo.	Motion		Overall	ExpOA
										4-CLS	Binary
Human Performance
Human	95.0	82.5	90.0	92.5	95.0	90.0	80.0	92.9	80.0	70.0	80.0	87.8	–
Baselines
Random guess	50.0	50.0	50.0	50.0	50.0	50.0	50.0	50.0	50.0	25.0	50.0	50.0	–
Proprietary MLLMs

Table 1: Results on 3DVerifierBench. All values represent the binary Balanced Accuracy, except for the 4-CLS column (four-class Exact Match accuracy). Bold marks the best model score.

Material	Object	Part	Aff. Reg.	Grasp	Ground.	Relation	Identity	FineGeo.	Motion	Overall
93.3	90.0	93.3	83.3	86.7	86.7	100.0	93.3	100.0	83.3	91.0

Table 2: Human validation of the explanation Judge. We sample 30 correctly rejected model responses per task. The numerical value in the bottom row is the Agreement rate (%), which represents the ratio of identical Judge and human assessments of explanation validity.

Panel of Figure 5 — Figure 5: Typical GPT-6 Astra failure cases and findings of limitations of MLLMs in 3D verification.

Model	Object	Part	Material	Avg.
Gemini-3.8	62.50 / 56.25	68.75 / 18.75	75.00 / 56.25	68.75 / 43.75
Gemma-4	62.50 / 37.50	62.50 / 50.00	68.75 / 37.50	64.58 / 41.67
Qwen3.5	68.75 / 50.00	62.50 / 50.00	62.50 / 50.00	64.58 / 50.00
Qwen3.6	56.25 / 50.00	68.75 / 50.00	68.75 / 50.00	64.58 / 50.00
Mean	62.50 / 48.44	65.63 / 42.19	68.75 / 48.44	65.63 / 46.35

Table 3: Evidence representation ablations. (a) Multi-view RGB v.s. raw 3D input. Each cell reports multi-view / raw OBJ Balanced Accuracy (%). (b) Motion verification under different evidence representations. Each cell reports binary / 4-class Acc.

Selector	Fact Sat.	Full	Strict
Random	80.8	26.7	16.6
GPT-6 Astra	90.4	55.0	45.0
Human Oracle	98.2	90.0	85.0

Table 4: What reliable verification could enable in 3D generation. Left: candidate selection from sixteen-candidate pools over 40 inputs. Right: human-guided recursive refinement on eight inputs. Full requires all seven facts; Strict additionally requires no severe defects and source fidelity.

Figure 6: Human-guided 3D recursive refinement trajectory. Given the source image and target facts, initial generation exhibits a missing side grille and a surface artifact (left, 6/7 facts). A revised one eliminates the artifact but leaves the grille incomplete (middle, 6/7 facts). Subsequent edit fully restores the mesh appearance and satisfies all facts (right, 7/7 facts). Intermediate 2D conditions and masks are omitted.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Task	GPT-6 Astra	GPT-5.6- Sol	Gemini-3.8- Flash	Gemma-4- 31B	Qwen3.5- 122B-A10B
Object	70.3%	70.3%	67.4%	68.6%	63.7%
Part	67.2%	62.3%	57.6%	50.7%	53.4%
Material	65.0%	58.1%	60.6%	60.0%	61.6%
Part Grounding	86.2%	81.1%	79.0%	64.4%	58.3%
Part Relation	88.0%	78.1%	84.9%	69.8%	66.1%
Affordance Region	73.1%	60.3%	72.1%	48.5%	48.4%

Appendix

Table E3: Explanation-aware Balanced Accuracy by task (%). The two blocks show complementary sets of five models.

Condition	GPT-6 Astra	GPT-5.6- Sol	Gemini-3.8- Flash	Gemma-4- 31B	Qwen3.5- 122B-A10B
Type 1	50.0%	50.0%	52.0%	52.0%	50.0%
Type 2	56.0%	50.0%	52.0%	50.0%	54.0%
Type 3	50.0%	50.0%	50.0%	52.0%	50.0%
Type 4	66.0%	52.0%	68.0%	72.0%	56.0%

Appendix

Table E5: Identity condition-wise ExpBA (%). The four conditions stratify examples by expression and texture nuisance settings.

Fine Fact	Units	GPT-6 Astra	GPT-5.6- Sol	Gemini-3.8- Flash	Gemma-4- 31B	Qwen3.5- 122B-A10B
identity_existence	59	78.0%	64.4%	72.0%	66.9%	65.3%
identity_preserving	200	55.5%	50.5%	55.5%	56.5%	52.5%
quantity	124	67.7%	65.3%	59.3%	53.6%	47.2%
shape_geometry	265	65.5%	62.5%	61.3%	55.3%	58.1%
spatial_configuration	94	83.5%	72.3%	78.7%	64.4%	58.5%
connectivity_contact	103	87.4%	82.5%	84.0%	76.2%	76.7%

Appendix

Table H3: Model-wise Fine Fact ExpBA (%). The two blocks show complementary sets of five models.

3D Scene Understanding

3D scene understanding builds geometric and semantic representations of environments for spatial reasoning, perception, and interaction.

See all 284 papers

3D-DefectBench: A Controlled Factorial Study of Vision-Language Model Evaluation Pipelines for Fine-Grained 3D Generation Defects
Jul 12, 2026Zhenyu Zhao, Nanshan Jia, Jihyeon Je +7VLM Evaluation
Roblox Corporation · Berkeley AI Research Lab & Department of Industrial Engineering and Operations Research, University of California, Berkeley · Computer Science Department, Stanford University +1
P3D-Bench: Benchmarking MLLMs for Parametric 3D Generation and Structural Reasoning
Jun 9, 2026Yikang Yang, Zhanpeng Hu, Youtian Lin +5Part-Level 3D Generation Parametric CAD Modeling
Nanjing University · Envision
BlenderWorld: Benchmarking Visual Agents for 3D Creation from Real-World References
Oct 14, 2026Xinyu Fang, Yuchi Wang, Weihao Yu +123D Scene Generation 3D Reconstruction Benchmarks
Zhejiang University · Alibaba Token Hub, Alibaba Group · The Chinese University of Hong Kong +2

Jul 12, 2026J/K move · Enter open · S save

3D-DefectBench: A Controlled Factorial Study of Vision-Language Model Evaluation Pipelines for Fine-Grained 3D Generation Defects

Zhenyu Zhao, Nanshan Jia, Jihyeon Je, Yifu Tang+6

Roblox Corporation · Berkeley AI Research Lab & Department of Industrial Engineering and Operations Research, University of California, Berkeley · Computer Science Department, Stanford University+1

VLM Evaluation

Open paper AskPDF arXiv

Automated evaluation is essential for scaling generative 3D systems, where exhaustive human review is costly and slow. Yet the reliability of an automated judge depends on the full evaluation pipeline, including the vision-language model (VLM), asset rendering, visual evidence, task specification, and human reference labels. We introduce 3D-DefectBench, a large-scale benchmark for rigorous evaluation-pipeline analysis. It complements holistic ratings and pairwise preferences with nine fine-grained binary defects spanning geometry, texture, and prompt adherence, with optional human severity annotations. Using a balanced factorial design, we vary the VLM, camera protocol, visual input, and prompt schema across 84 inference designs, and validate the resulting conclusions on a broader set of frontier models. Model choice is the dominant source of variation in agreement with human labels, while other pipeline factors also influence agreement, interact with the model, and can alter the best configuration. A compact six-view RGB protocol performs comparably to denser view sets and configurations augmented with depth or normal channels, making it a strong cost-effective default. Under this fixed design, the best of 12 VLMs still trail trained human labelers, and texture agreement drops sharply from expert-agreement to noisier silver labels. Severity annotations further show that binary judges recover most defects humans flag as severe. These results highlight the importance of evaluating automated judges as complete pipelines and calibrating them across human reference regimes.

Abstract

3D Scene Understanding

Explore similar work

3D-DefectBench: A Controlled Factorial Study of Vision-Language Model Evaluation Pipelines for Fine-Grained 3D Generation Defects

P3D-Bench: Benchmarking MLLMs for Parametric 3D Generation and Structural Reasoning

BlenderWorld: Benchmarking Visual Agents for 3D Creation from Real-World References