PhyMix: Towards Physically Consistent Single-Image 3D Indoor Scene Generation with Implicit--Explicit Optimization

Dongli Wu; Jingyu Hu; Ka-Hei Hui; Xiaobao Wei; Chengwen Luo; Jianqiang Li; Zhengzhe Liu

cs.CVApr 11, 2026

PhyMix: Towards Physically Consistent Single-Image 3D Indoor Scene Generation with Implicit--Explicit Optimization

Authors: Dongli Wu, Jingyu Hu, Ka-Hei Hui, Xiaobao Wei, Chengwen Luo, Jianqiang Li, Zhengzhe Liu

Organizations: 1Lingnan University · 2The Hong Kong University of Science and Technology (Guangzhou) · 3The Chinese University of Hong Kong · 4Peking University ·

Abstract

Existing methods for single-image 3D indoor scene generation often produce visually plausible results that violate real-world physical constraints, limiting their reliability in robotics, embodied AI, and virtual environment design. To bridge this gap, we introduce a Physics Evaluator that assesses four aspects (geometric priors, contact, stability, and deployability) through nine sub-constraints, establishing a benchmark for evaluating physical consistency. Our analysis reveals that even state-of-the-art methods produce scenes with substantial physical inconsistencies despite their visual plausibility. To address these limitations, we propose a framework that incorporates feedback from the Physics Evaluator into both training and inference to improve the physical plausibility of generated scenes. Specifically, PhyMix comprises two components: (i) implicit alignment via Scene-GRPO, a critic-free group-relative preference optimization strategy that uses physical evaluations to guide the generator toward physically plausible layouts, and (ii) explicit refinement via verifier-guided agentic repair, which translates evaluator diagnostics into executable scene-editing skills to correct residual violations at inference time. PhyMix unifies physical evaluation, preference-based alignment, and agentic inference-time correction to improve physical plausibility while retaining the observed scene content. Experiments show improvements in reconstruction quality and physical plausibility, while qualitative examples on stylized and real-world images illustrate the applicability of the framework beyond synthetic inputs.

Figures & tables

Panel of Table 1 — Table 1: Coverage of physical scene properties in existing methods.

Figure 3: Overview of PhyMix. (a) The Physics Evaluator measures geometric priors, contact, stability, and deployability. (b) Scene-GRPO aligns the generator through candidate generation, flow-matching likelihood scoring, physical preferences, and group-relative optimization. (c) Verifier-guided agentic repair corrects residual violations through diagnosis, skill proposals, testing edits on scene copies, and verified updates. Committed edits and candidate verification outcomes supervise the local repair policy.

(a) Cross-backbone
Backbone	Added modules	Base $\uparrow$	Updated $\uparrow$
MIDI	GRPO + Agentic	85.3	98.6
PartCrafter	GRPO + Agentic	88.8	96.3
PAT3D	Agentic only	93.2	97.1

Table 5: Backbone, candidate-group, and reference-regularization comparisons. In (a), MIDI and PartCrafter use GRPO alignment and agentic repair, while PAT3D uses agentic repair only, without generator alignment. Parts (b) and (c) evaluate the full pipeline. Relative time is normalized to the

K=2

configuration.

Figure 5: Stage-wise comparison: (a) base, (b) Scene-GRPO, and (c) verifier-guided agentic repair. Red boxes highlight problematic regions.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Test	Checked objects	Fails if
Orientation	Objects whose category has an upright direction	tilt exceeds $\theta_{o}=15^{\circ}$
Scale	Objects whose category has $\geq$ 20 training examples	a dimension lies outside $[0.8Q_{0.05},\,1.2Q_{0.95}]$ for the category
Collision	All object pairs and object–room pairs	overlap ratio $>\eta_{c}=0.005$ and penetration $>\tau_{c}=0.001L$
Grounding	Objects assigned to the floor	gap $>\tau_{g}=0.002L$ , penetration $>\tau_{c}$ , or contact area $<1\%$ of the footprint
Support	Objects assigned to another object	the supporting object is missing, or the gap, penetration, or contact-area check fails
Anchoring	Objects assigned to a wall or ceiling	mounting gap $>\tau_{a}=0.002L$ or mounting angle error $>\theta_{a}=10^{\circ}$

Appendix

Table 11: The nine physical tests. A violation rate is the fraction of checked objects (or pairs) that fail.

L

is defined in equation 19 .

Component	Settings
Optimizer	AdamW; learning rate $10^{-5}$ ; betas $(0.9,0.999)$ ; weight decay 0.01; gradient clipping 1.0
Training	10,000 updates; batch 4 with 8 accumulation steps (effective batch 32)
LoRA	Rank 16, scale 32, dropout 0.05, on layout-conditioning attention and pose heads; geometry encoder and decoder frozen
Candidates per input	12: six from the current model and six perturbed samples from the EMA model (decay 0.999)
Perturbations	Translation $\leq 0.02L$ , rotation $\leq 10^{\circ}$ , log-scale change $\leq 0.05$
Objective	$\tau_{p}=\tau_{r}=1$ ; advantage clip $a=3$ ; $\lambda_{\rm align}=\lambda_{\rm FM}=1$ ; KL weight from 0.1 to 0.01 ( equation 25 )

Appendix

Table 12: Training settings of the generator (top) and the repair policy (bottom). EMA: exponential moving average.

Edit	What it does
Translate	Moves an object by at most $0.02L$ per edit and $0.1L$ in total
Rotate	Rotates an object about its contact with its support, by at most $15^{\circ}$ per edit and $30^{\circ}$ in total
Rescale	Changes an object’s log-scale by at most 0.05 per edit (about 5%) and 0.15 in total
Separate collision	Pushes two overlapping objects apart along the shortest direction
Snap to support	Moves an object straight down or up onto its support surface
Align to wall	Moves a wall- or ceiling-mounted object back onto its mounting surface

Appendix

Table 13: The seven repair edits. Totals are measured from the initial scene

S_{0}

. The limits of the first three rows apply to every edit.

Image-to-3D Scene Generation

See all 24 papers

PhyScene3D: Physically Consistent Interactive 3D Tabletop Scene Generation
Jun 1, 2026Weixing Chen, Zhuoqian Feng, Yang Liu +63D Scene Generation Functional 3D Scene Generation
Sun Yat-sen University, China · Guangdong Key Laboratory of Big Data Analysis and Processing · Huawei +1
TelePhysics: Physics-Grounded Multi-Object Scene Generation from a Single Image with Real-Time Interaction
May 19, 2026Xin Zhang, Yabo Chen, Yijie Fang +5Physical Consistency in Video Generation Real-Time Interactive Video Generation
Fudan University · Institute of Artificial Intelligence, China Telecom (TeleAI)
Tetris3D: 3D Scene Generation With Objects That Fit Together
Oct 7, 2026Jaeyeong Kim, Jinhyuk Jang, Jongmin Lee +23D Scene Generation Image-to-3D Scene Generation
KAIST AI

Jun 1, 2026J/K move · Enter open · S save

PhyScene3D: Physically Consistent Interactive 3D Tabletop Scene Generation

Weixing Chen, Zhuoqian Feng, Yang Liu, Yexin Zhang+5

Sun Yat-sen University, China · Guangdong Key Laboratory of Big Data Analysis and Processing · Huawei+1

3D Scene Generation Functional 3D Scene Generation 3D Scene Layout Generation

Open paper AskPDF arXiv

Generating physically consistent 3D tabletop scenes is a fundamental yet underexplored problem for interactive and generalist robotic learning. The challenge stems from dense object hierarchies and irregular affordances. Here, an interactive scene denotes a physically valid, collision-free environment directly loadable into physics simulators. Existing methods, ranging from decoupled symbolic solvers to end-to-end regression models, often suffer from error propagation or overfitting to noisy supervision containing widespread physical violations. To address these limitations, we introduce PhyScene3D, a framework that reformulates generation as a Human-Mimetic Constructive Process. The proposed Cognitive Topological Reasoning Chain (CTRC) factorizes scene synthesis into a sequential, anchor-conditioned process. It employs a 3D AABB-based placement scheme that imposes a strong structural inductive bias. To address imperfect supervision and physical infeasibility, we introduce Physics-Aware Denoising Alignment (PADA). It integrates a differentiable Signed Distance Field (SDF) with Test-Time Optimization (TTO) to project generated scenes onto a physics-feasible manifold while preserving semantic intent. Experiments demonstrate that PhyScene3D outperforms state-of-the-art approaches in both semantic accuracy and physical validity, achieving a 40% reduction in scene-wise collision rate relative to the human-annotated training data.

Abstract

Image-to-3D Scene Generation

Explore similar work

PhyScene3D: Physically Consistent Interactive 3D Tabletop Scene Generation

TelePhysics: Physics-Grounded Multi-Object Scene Generation from a Single Image with Real-Time Interaction

Tetris3D: 3D Scene Generation With Objects That Fit Together