RelaxFlow: Text-Driven Amodal 3D Generation

Jiayin Zhu; Guoji Fu; Xiaolu Liu; Qiyuan He; Yicong Li; Angela Yao

cs.CVMar 5, 2026

RelaxFlow: Text-Driven Amodal 3D Generation

Authors: Jiayin Zhu, Guoji Fu, Xiaolu Liu, Qiyuan He, Yicong Li, Angela Yao

Organizations: National University of Singapore · Zhejiang University · University of Science and Technology of China

Abstract

Image-to-3D generation faces inherent semantic ambiguity under occlusion, where partial observation alone is often insufficient to determine object category. In this work, we formalize text-driven amodal 3D generation, where text prompts steer the completion of unseen regions while strictly preserving input observation. Crucially, we identify that these objectives demand distinct control granularities: rigid control for the observation versus relaxed structural control for the prompt. To this end, we propose RelaxFlow, a training-free dual-branch framework that decouples control granularity via a Multi-Prior Consensus Module and a Relaxation Mechanism. Theoretically, we prove that our relaxation is equivalent to applying a low-pass filter on the generative vector field, which suppresses high-frequency instance details to isolate geometric structure that accommodates the observation. To facilitate evaluation, we introduce two diagnostic benchmarks, ExtremeOcc-3D and AmbiSem-3D. Extensive experiments demonstrate that RelaxFlow successfully steers the generation of unseen regions to match the prompt intent without compromising visual fidelity.

Figures & tables

Figure 1: The case for multiple plausible amodal 3D interpretations under occlusion. Feedforward image-to-3D model ( e.g . , SAM3D) collapses to a single overfitted bed-like shape, while our RelaxFlow resolves the ambiguity via text-driven amodal generation , allowing users to inject explicit textual intent and steer the generation toward alternative semantically consistent amodal 3D shapes.

Figure 2: Conceptual illustration of low-pass relaxation. The background depicts a conceptual compatibility landscape over latent states, where high density corresponds to low “energy”. Star markers indicate a spurious trapped mode and the intended target mode. Throughout, we use corridor to denote a tube of latent states traced by integral curves of a conditioned velocity field while satisfying a constraint. Smoothing the prior-conditioned guidance thickens the semantic corridor and steers trajectories toward the intended mode while remaining compatible with the observation corridor.

Figure 3: RelaxFlow pipeline overview . An observation-driven branch preserves visible evidence, while the semantic guidance is injected through the intent prompt via multi-prior consensus and low-pass relaxation. Dual branches are fused via velocity blending to resolve occlusion-induced ambiguity.

Figure 4: Qualitative Comparisons. Top: Comparisons on AmbiSem-3D examples. Each contains two different intent prompts for the same observation. Our method preserves observation fidelity while enabling prompt-controlled completion. Bottom: The case in ExtremeOcc-3D with an intent prompt “bed”. Under extreme occlusion, baselines either overfit the visible region or yield implausible shapes, whereas ours follows the category prior while maintaining visible evidence.

Category	Method	CLIP ${}_{\text{img}}$ $\uparrow$	CLIP ${}_{\text{txt}}$ $\uparrow$	FID $\downarrow$	LPIPS $\downarrow$	Point-FID $\downarrow$
Non-Occlusion Aware	TRELLIS	0.78	23.14	122.68	0.83	141.48
Non-Occlusion Aware	Ours (w/ TRELLIS)	0.80	24.09	100.75	0.80	97.79
Occlusion- Aware	Amodal2D+SAM3D	0.76	21.59	94.38	0.56	127.27
	Amodal3R	0.77	22.29	118.49	0.60	129.46
	SAM3D	0.84	24.08	50.73	0.54	100.38
	Ours (w/ SAM3D)	0.87	27.26	39.44	0.51	81.11

Table 1: Quantitative comparison on ExtremeOcc-3D. Methods are grouped by occlusion awareness. Our method improves over both backbones across all metrics.

Method	CLIP Score		User Study
	CLIP ${}_{\text{img}}$ $\uparrow$	CLIP ${}_{\text{txt}}$ $\uparrow$	Alignment $\uparrow$	3D Fidelity $\uparrow$	Overall Pref. $\uparrow$
SAM3D	0.85	26.29	4.84%	13.59%	9.22%
TRELLIS (multi-view)	0.80	26.59	3.75%	8.28%	6.02%
SDXL + TRELLIS	0.81	26.76	6.09%	8.91%	7.50%
SDXL + SAM3D	0.79	26.71	11.41%	6.09%	8.75%
Ours	0.87	27.23	73.91%	63.13%	68.52%

Table 2: Quantitative comparison on AmbiSem-3D. Left: automatic metrics (CLIP similarities). Right: user study results (

n{=}32

). Our method achieves the best scores on both evaluations.

Figure 5: Visual ablation results. Top: effect of key components. Bottom: effect of varying the number of prior images $N$ .

Panel of Figure 6 — Figure 6: Ablation studies on ExtremeOcc-3D with SAM3D backbone. (a) Component ablation and hyperparameter sensitivity. Removing low-pass relaxation or visibility mask degrades performance; extreme $\rho$ or $\sigma$ values also hurt. “Prior from generation” uses Z-Image-generated priors instead of retrieved ones. (b) Effect of prior count $N$ : moderate $N$ improves consensus, but too many priors introduce conflicts.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Visualization of all cases in AmbiSem-3D.

Figure 8: More examples of cases in ExtremeOcc-3D.

Method	CLIP ${}_{\text{img}}$ $\uparrow$	CLIP ${}_{\text{txt}}$ $\uparrow$	FID $\downarrow$	LPIPS $\downarrow$	Point-FID $\downarrow$
Wan2.2-TI2V + TRELLIS	0.78	22.98	107.49	0.85	110.3
MV-Adapter + TRELLIS	0.80	22.70	148.66	0.81	238.3
TRELLIS (text+image)	0.82	26.36	116.51	0.79	137.7
Nano Banana Pro + 3D	0.83	27.49	77.68	0.77	88.6
Ours	0.87	27.26	39.44	0.51	81.1

Appendix

Table 3: Comparison with other baseline families on ExtremeOcc-3D. These baselines cover video generation, multi-view generation, direct text+image conditioning, and commercial 2D editing pipelines.

Method	CLIP ${}_{\text{img}}$ $\uparrow$	CLIP ${}_{\text{txt}}$ $\uparrow$
Wan2.2-TI2V + TRELLIS	0.80	26.71
MV-Adapter + TRELLIS	0.81	25.54
TRELLIS (text+image)	0.81	27.24
Nano Banana Pro + 3D	0.81	27.41
Ours	0.87	27.23

Appendix

Table 4: Comparison with other baseline families on AmbiSem-3D. Because AmbiSem-3D has multiple valid completions and no unique 3D ground truth, we report CLIP-based observation and text alignment.

Figure 9: Qualitative comparison with the TRELLIS (text+image) fusion baseline. This baseline often produces implausible geometry because the objects generated by its text- and image-conditioned branches are not visually consistent, making spatial alignment difficult. In contrast, our method injects text conditioning as low-pass semantic guidance in the visual latent space, yielding more coherent and geometrically consistent results.

Figure 10: Qualitative comparison with Wan2.2-TI2V + TRELLIS and MV-Adapter + TRELLIS. For Wan2.2-TI2V, we use the enhanced prompt: “A clean studio turnaround video of intent prompt, centered in frame, isolated from the background, with the camera orbiting smoothly around the object to reveal multiple viewpoints. The object stays fixed while only the camera moves. Consistent geometry, consistent appearance, product-shot lighting, plain neutral background, the full object remains visible in every frame, no extra objects, no scene changes, no cuts, no zoom.” Despite this careful prompting, the video-generation pipeline often fails to produce 3D-consistent frames, leading to distorted or blurred 3D reconstructions. The multi-view generation pipeline also struggles in this setting, often failing to preserve the observation evidence while following the text prompt, which results in distorted outputs. In contrast, our method operates during 3D generation and produces results that are both geometrically consistent and faithful to the input observation and text intent.

Figure 11: Qualitative comparison with Nano Banana Pro + 3D. As a non-3D-aware commercial 2D editing model, Nano Banana Pro can produce edits with implausible geometry, back-view inconsistency, and attribute leakage. In contrast, our method achieves text-guided amodal 3D generation that remains consistent with both the text intent and the observation evidence.

Method	CLIP ${}_{\text{img}}$ $\uparrow$	CLIP ${}_{\text{txt}}$ $\uparrow$
SAM3D	0.839	25.6
Ours	0.845	26.9

Appendix

Table 5: Extended 100-case AmbiSem-3D evaluation.

Backbone	Method	Time (s/sample)	Allocated (GB)	Reserved (GB)
SAM3D	Base	15.59	18.07	21.85
	Ours	38.40	18.60	22.60
	Overhead	2.46 $\times$	+2.9%	+3.4%
TRELLIS	Base	12.81	5.57	5.60
	Ours	23.31	6.02	6.07
	Overhead	1.82 $\times$	+8.0%	+8.4%

Appendix

Table 6: Runtime and memory overhead. Memory is peak GPU memory in GB. RelaxFlow adds extra branch evaluations, increasing runtime, while peak memory grows modestly. The current implementation prioritizes memory over runtime; batched branch inference and feature caching may further reduce wall-clock overhead.

Figure 12: More qualitative results. TRELLIS and SAM3D remain overfitted to the observed geometry, producing either a shield or a TIE fighter regardless of intent, whereas our method generates either structure as specified by the intent prompt.

Figure 15: More qualitative results on ExtremeOcc-3D.

Figure 16: Failure cases under conflicting conditions. When the observation and prior disagree, no evidence-consistent solution exists, and our method may interpolate between them as a soft concept fusion due to asymmetric guidance.

Text-to-3D Generation

Text-to-3D generation creates 3D objects and scenes from language while balancing semantic alignment, geometry, appearance, and spatial coherence.

See all 64 papers

SpaceFlow: Locally Controllable 3D Generation
Oct 8, 2026Neil De La Fuente, Joan Lafuente, Mukhammadali Sayfiddinov +53D Shape Generation Text-to-3D Generation
ETH Zürich · Microsoft · Stanford University
GENA3D: Generative Amodal 3D Modeling by Bridging 2D Priors and 3D Coherence
Nov 26, 2025Junwei Zhou, Yu-Wing Tai3D Reconstruction Generative Modeling
Dartmouth College
SeeThrough3D: Occlusion Aware 3D Control in Text-to-Image Generation
Feb 26, 2026Vaibhav Agrawal, Rishubh Parihar, Pradhaan Bhat +2Controllable Image Generation Occlusion Reasoning
IIIT Hyderabad · IISc Bengaluru

Oct 8, 2026J/K move · Enter open · S save

SpaceFlow: Locally Controllable 3D Generation

Neil De La Fuente, Joan Lafuente, Mukhammadali Sayfiddinov, Felicia Scharitzer+4

ETH Zürich · Microsoft · Stanford University

3D Shape Generation Text-to-3D Generation Part-Level 3D Generation

Open paper AskPDF arXiv

Current 3D generation methods lack explicit local control: geometric adherence is often defined by a global control strength, and appearance cannot be specified locally. We present SpaceFlow, a training-free pipeline for locally controllable 3D generation from text descriptions and a collection of geometric primitives. Each primitive serves as a proxy for an object part and is assigned a local control level, enabling users to specify whether regions should strictly follow the input shape or allow generative completion. During structure generation, we enforce these spatial constraints within the generative flow process. For appearance synthesis, the generated structure is segmented and matched to the primitives. Each generated part is conditioned only on its assigned text or image cue, thereby limiting cross-part leakage. Regional geometry metrics demonstrate that SpaceFlow preserves the specified geometry in high-control regions and enables plausible shape variation in low-control areas. A user study further indicates that the resulting balance between geometric fidelity and generative freedom remains competitive in overall quality. When evaluating appearance on fixed geometry, text-conditioned routing achieves state-of-the-art prompt faithfulness and color/material accuracy. Qualitative results additionally show localized routing of image cues. The project page is available at SpaceFlow3D.github.io.

Abstract

Text-to-3D Generation

Explore similar work

SpaceFlow: Locally Controllable 3D Generation

GENA3D: Generative Amodal 3D Modeling by Bridging 2D Priors and 3D Coherence

SeeThrough3D: Occlusion Aware 3D Control in Text-to-Image Generation