cs.CVNov 8, 2026

SurfScaff3D: Bridging Geometric and Semantic Representations for LLM-based 3D Object Generation

Authors: Anay Loya, Rishi Gupta, Arnav Samal, Anirban Chakraborty

Organizations: Indian Institute of Science, Bengaluru, India

Abstract

Creating high-quality 3D content has traditionally required specialist expertise and experience with complex tools. The growing ecosystem of AI-assisted creativity demands accessible and intuitive 3D generation systems. Recent advances in Large Language Models (LLMs) offer promising directions, yet extending their generative capabilities to 3D remains fundamentally challenging because existing 3D representations are not naturally compatible with language-based reasoning. Current approaches either depend on expensive fine-tuning or impose restrictive structural priors that sacrifice expressiveness and exclude organic shapes. We argue that the mismatch between LLM-native reasoning and conventional 3D parameterizations is a core bottleneck of LLM-driven 3D generation. To address this, we introduce the Hierarchical Patch Scaffold (HPS), a representation designed around semantically meaningful, interpretable part-level abstractions that are naturally understood by both humans and LLMs, enabling expressive, editable, and open-domain 3D generation without imposing restrictive geometric priors or expensive fine-tuning. Building upon HPS, we introduce SurfScaff3D, an interactive framework for text and image-to-3D generation. HPS inherits the flexibility of B-spline patches, while exposing an interface that leverages open-world capabilities of foundation models to reliably generate 3D. Its hierarchical part-level semantic labels enable targeted, natural-language edits at any level of granularity without re-running generation. Further, HPS offers explicit per-patch control over surface detail, enabling more expressive geometry capable of representing arbitrary objects. Extensive experiments against state-of-the-art LLM-based 3D generation methods demonstrate strong geometric fidelity, semantic alignment, and flexible editing across diverse object categories.

Figures & tables

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Let Language Constrain Geometry: Vision-Language Models as Semantic and Spatial Critics for 3D Generation

    Nov 18, 2025Weimin Bai, Yubo Li, Weijian Luo +43D Spatial ReasoningText-to-3D Generation

  2. DreamPartGen: Semantically Grounded Part-Level 3D Generation via Collaborative Latent Denoising

    Mar 19, 2026Tianjiao Yu, Xinzhuo Li, Muntasir Wahed +4Part-Level 3D Generation3D Shape Generation

  3. NaLA: A 3D Native LLM Layout Agent for High-quality 3D Scene Generation

    Jun 28, 2026Cheng Wan, Yongsen Mao, Wenzheng Wu +73D Spatial Reasoning3D Scene Generation