Organizations: 1Lingnan University · 2The Hong Kong University of Science and Technology (Guangzhou) · 3The Chinese University of Hong Kong · 4Peking University · 5Shenzhen University
Existing methods for single-image 3D indoor scene generation often produce visually plausible results that violate real-world physical constraints, limiting their reliability in robotics, embodied AI, and virtual environment design. To bridge this gap, we introduce a Physics Evaluator that assesses four aspects (geometric priors, contact, stability, and deployability) through nine sub-constraints, establishing a benchmark for evaluating physical consistency. Our analysis reveals that even state-of-the-art methods produce scenes with substantial physical inconsistencies despite their visual plausibility. To address these limitations, we propose a framework that incorporates feedback from the Physics Evaluator into both training and inference to improve the physical plausibility of generated scenes. Specifically, PhyMix comprises two components: (i) implicit alignment via Scene-GRPO, a critic-free group-relative preference optimization strategy that uses physical evaluations to guide the generator toward physically plausible layouts, and (ii) explicit refinement via verifier-guided agentic repair, which translates evaluator diagnostics into executable scene-editing skills to correct residual violations at inference time. PhyMix unifies physical evaluation, preference-based alignment, and agentic inference-time correction to improve physical plausibility while retaining the observed scene content. Experiments show improvements in reconstruction quality and physical plausibility, while qualitative examples on stylized and real-world images illustrate the applicability of the framework beyond synthetic inputs.
Figures & tables
Table 1: Coverage of physical scene properties in existing methods.
Figure 3: Overview of PhyMix. (a) The Physics Evaluator measures geometric priors, contact, stability, and deployability. (b) Scene-GRPO aligns the generator through candidate generation, flow-matching likelihood scoring, physical preferences, and group-relative optimization. (c) Verifier-guided agentic repair corrects residual violations through diagnosis, skill proposals, testing edits on scene copies, and verified updates. Committed edits and candidate verification outcomes supervise the local repair policy.
(a) Cross-backbone
Backbone
Added modules
Base ↑
Updated ↑
MIDI
GRPO + Agentic
85.3
98.6
PartCrafter
GRPO + Agentic
88.8
96.3
PAT3D
Agentic only
93.2
97.1
Table 5: Backbone, candidate-group, and reference-regularization comparisons. In (a), MIDI and PartCrafter use GRPO alignment and agentic repair, while PAT3D uses agentic repair only, without generator alignment. Parts (b) and (c) evaluate the full pipeline. Relative time is normalized to the K=2 configuration.
Figure 5: Stage-wise comparison: (a) base, (b) Scene-GRPO, and (c) verifier-guided agentic repair. Red boxes highlight problematic regions.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Test
Checked objects
Fails if
Orientation
Objects whose category has an upright direction
tilt exceeds θo=15∘
Scale
Objects whose category has ≥ 20 training examples
a dimension lies outside [0.8Q0.05,1.2Q0.95] for the category
Collision
All object pairs and object–room pairs
overlap ratio >ηc=0.005 and penetration >τc=0.001L
Grounding
Objects assigned to the floor
gap >τg=0.002L , penetration >τc , or contact area <1% of the footprint
Support
Objects assigned to another object
the supporting object is missing, or the gap, penetration, or contact-area check fails
Anchoring
Objects assigned to a wall or ceiling
mounting gap >τa=0.002L or mounting angle error >θa=10∘
Appendix
Table 11: The nine physical tests. A violation rate is the fraction of checked objects (or pairs) that fail. L is defined in equation 19 .
Generating physically consistent 3D tabletop scenes is a fundamental yet underexplored problem for interactive and generalist robotic learning. The challenge stems from dense object hierarchies and irregular affordances. Here, an interactive scene denotes a physically valid, collision-free environment directly loadable into physics simulators. Existing methods, ranging from decoupled symbolic solvers to end-to-end regression models, often suffer from error propagation or overfitting to noisy supervision containing widespread physical violations. To address these limitations, we introduce PhyScene3D, a framework that reformulates generation as a Human-Mimetic Constructive Process. The proposed Cognitive Topological Reasoning Chain (CTRC) factorizes scene synthesis into a sequential, anchor-conditioned process. It employs a 3D AABB-based placement scheme that imposes a strong structural inductive bias. To address imperfect supervision and physical infeasibility, we introduce Physics-Aware Denoising Alignment (PADA). It integrates a differentiable Signed Distance Field (SDF) with Test-Time Optimization (TTO) to project generated scenes onto a physics-feasible manifold while preserving semantic intent. Experiments demonstrate that PhyScene3D outperforms state-of-the-art approaches in both semantic accuracy and physical validity, achieving a 40% reduction in scene-wise collision rate relative to the human-annotated training data.
Weixing Chen, Zhuoqian Feng, Yang Liu +6
Sun Yat-sen University, China · Guangdong Key Laboratory of Big Data Analysis and Processing · Huawei +1
Recent generative video models achieve impressive visual quality but remain constrained by limited physical consistency and controllability. Existing video generation methods provide minimal physical control, and single-image-to-3D conversion approaches often suffer from object interpenetration. Furthermore, physics-based scene-level 3D generation methods exhibit spatial misalignment, stylized artifacts, and inconsistencies with the input data, restricting their use in realistic interactive video synthesis. We propose TelePhysics, a training-free framework that converts a single image into a physically consistent and controllable video through holistic scene-level 3D reconstruction. By representing the full scene geometry in a unified spatial coordinate system, TelePhysics resolves object penetration and alignment ambiguity. Unlike prior methods, this formulation enables accurate scenelevel multi-object interactions and introduces richer, complex control types for advanced mechanicsbased manipulation. By decoupling simulation from rendering, TelePhysics bypasses latency-heavy priors, achieving real-time physical interaction previews paired while preserving photorealistic visual fidelity. Experimental results demonstrate that TelePhysics substantially outperforms prior methods in physical fidelity, spatial coherence, and controllability. The open-source code is available at https://github.com/xinzhang007/TelePhysics.
Xin Zhang, Yabo Chen, Yijie Fang +5
Fudan University · Institute of Artificial Intelligence, China Telecom (TeleAI)
We propose Tetris3D, a generative framework for single-image 3D scene reconstruction that recovers objects which are physically and geometrically coherent as a scene. Existing methods often generate objects independently or couple them implicitly, providing limited guidance for ensuring fine-grained spatial compatibility between neighboring objects that interact with one another. To address this, we explicitly condition the generation of each object on the geometry of surrounding objects and their physical relationships, guiding its shape and pose to remain geometrically and physically plausible within the scene. Moreover, we introduce ComOb, a physics simulation-based dataset of 1.2M scenes featuring physical interactions across diverse object categories, with per-object meshes and pairwise physical relation annotations. Comprehensive experiments on synthetic and realworld scenes show that Tetris3D recovers coherent object shapes and poses even when interacting regions are occluded, and achieves state-of-the-art performance in both generation quality and physical stability.