Self-Consistency Geometry Injection for Multi-view Consistent Image Generation

Yupeng Xiao; Yankai Wu; Jianjie Luo; Jiajie Lin; Yi Yu; Zhenguo Yang

cs.CVOct 28, 2026

Self-Consistency Geometry Injection for Multi-view Consistent Image Generation

Authors: Yupeng Xiao, Yankai Wu, Jianjie Luo, Jiajie Lin, Yi Yu, Zhenguo Yang

Organizations: Guangdong University of Technology, China · Hiroshima University, Japan

Abstract

Recent diffusion models have achieved impressive progress in single image to multi-view generation by synthesizing consistent novel views efficiently. Existing methods usually explicitly or implicitly couple geometry inference and texture synthesis, making generation vulnerable to inaccurate geometry prediction. Although several geometry-guided methods have explored decoupling, they usually treat predicted geometry as a fixed condition and do not explicitly model whether the geometry is reliable. We therefore propose Reliability-Guided Multi-View Generation (RGMV), which comprises a parameter-free Self-Consistency Geometry Gate (SCG) and a Reliability-aware Geometry Injection (RGI) module. Specifically, a geometry proposal generator produces multiple geometry predictions for the same reference image and target viewpoints under different stochastic sampling configurations. SCG measures their spatial agreement to derive a reliability map, which RGI uses to modulate the injection strength of geometry-conditioned residuals in the RGB synthesis model. Moreover, RGMV can adapt to different geometry proposal generators. As a result, RGMV preserves stronger geometric guidance in stable regions while attenuating unreliable geometry cues before they propagate into multi-view RGB generation. Extensive experiments show that reliability-aware geometry injection improves the geometric quality of multi-view generation. Code: https://github.com/WISLab-GDUT/RGMV.

Figures & tables

Figure 1: Comparison of three paradigms for multi-view image generation. (a) Direct generation entangles geometric reasoning and texture synthesis in a single generator, making geometric errors difficult to isolate. (b) Geometry-guided generation separates geometry prediction from RGB synthesis, but applies predicted geometry with spatially uniform influence. (c) RGMV conditions RGB synthesis on an anchor geometry prediction and uses per-view spatial reliability maps estimated by SCG to modulate geometry residuals through RGI.

Figure 2: Overview of RGMV. Given a reference image and target camera poses, an off-the-shelf geometry proposal generator produces $K$ stochastic sets of multi-view normal predictions. The first prediction set serves as the anchor normal condition to produce multi-level geometry residuals. In parallel, Self-Consistency Geometry Gate (SCG) in (a) groups normal predictions by target view, measures their per-pixel angular disagreement, and converts it into per-view spatial reliability maps. Reliability-aware Geometry Injection (RGI) module in (b) resizes and affinely calibrates the corresponding reliability map at each denoising level $l$ to obtain $\widetilde{\mathbf{R}}_{v,l}$ , which selectively modulates the geometry residual before it is injected into the multi-view diffusion backbone.

Method	Multi-view image generation			3D reconstruction
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	CD $\downarrow$	IoU $\uparrow$
Zero-1-to-3	17.38	0.8479	0.1383	0.071	0.33
SyncDreamer	18.83	0.8212	0.2146	0.066	0.39
Wonder3D++	20.05	0.8575	0.1463	0.062	0.44
MV-Adapter	20.13	0.8664	0.1268	0.036	0.54
Era3D	20.77	0.8634	0.1253	0.038	0.53

Table 1: Comparison of RGB synthesis and 3D reconstruction on GSO. All RGMV variants share the same adapted RGB synthesis model; parentheses indicate the geometry proposal generator. CD denotes Chamfer Distance, and IoU denotes Volume IoU.

Figure 3: Qualitative comparison on rendered objects with ground-truth target views. Each row shows the multi-view results of one method under the same target viewpoints.

Figure 4: Qualitative comparison on real captured images without ground-truth target views. The examples are organized by representative failure modes commonly observed in multi-view generation. Each pair compares a baseline result with its corresponding RGMV result under the same input and target viewpoints.

Statistic	Result	Range
Pooled Spearman $\rho$	0.299	$[-1,1]$
Per-sample $\rho$ ( $\mu/\mathrm{med}/\sigma$ )	0.266 / 0.289 / 0.166	$[-1,1]$
AUROC ( $e>20^{\circ}/30^{\circ}$ )	0.729 / 0.710	$[0,1]$

Table 2: Association between SCG reliability and ground-truth normal error on Objaverse with

K=3

Figure 5: Sensitivity of spatially uniform geometry injection on Objaverse with a frozen RGB synthesis model. The dashed line marks

\alpha_{\mathrm{uni}}=0.1

, which is selected for the uniform-injection baseline. Larger global scales progressively degrade performance.

Setting	Injection	RGB Model	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Pure i2mv	–	Frozen	24.22	0.8477	0.2069
Uniform Injection	Uniform	Frozen	24.43	0.8498	0.2073
SCG Injection	SCG	Frozen	24.63	0.8545	0.1935
Uniform Injection	Uniform	Adapted	25.14	0.8569	0.1948
RGMV	SCG	Adapted	25.31	0.8621	0.1823

Table 3: Controlled ablation of predicted geometry conditioning, reliability-aware injection, and multi-view RGB synthesis model adaptation. “Uniform” denotes spatially constant geometry residual injection with

\alpha_{\mathrm{uni}}=0.1

, selected by the sweep in Fig. 5 .

$K$	proposal generation (s)	SCG computation (s)	RGB synthesis (s)	Total (s)
1	$3.84\pm 0.02$	$0.053\pm 0.001$	$4.95\pm 0.01$	$8.85\pm 0.02$
2	$7.71\pm 0.02$	$0.089\pm 0.002$	$4.96\pm 0.01$	$12.76\pm 0.02$
3	$11.56\pm 0.02$	$0.124\pm 0.003$	$4.97\pm 0.01$	$16.65\pm 0.02$
4	$15.41\pm 0.02$	$0.163\pm 0.002$	$4.96\pm 0.01$	$20.54\pm 0.03$

Table 5: Inference time on an RTX 4090 GPU for different geometry proposal sets

K

Multi-View Image Synthesis

See all 29 papers

GeoFace: Consistent Multi-View Face Generation with Geometry-Constrained Diffusion
Jun 26, 2026Yeji Choi, Jinhyeok Choi, Jaewon Min +3Multi-View Consistency 3D Face Reconstruction
KAIST AI
VGGT-Diff: Visual Geometry Meets Diffusion for Sparse-View Novel View Synthesis
Sep 27, 2026Kangjie Chen, Xiangyu Li, Dongbin Zhang +12Novel View Synthesis Conditional Diffusion Models
XPeng Motors · The Chinese University of Hong Kong · Tsinghua University
Repurposing Geometric Foundation Models for Multi-view Diffusion
Mar 23, 2026Wooseok Jang, Seonghu Jeon, Jisang Han +5Multi-View Diffusion Models Novel View Synthesis
KAIST AI · New York University · Intel Labs

Jun 26, 2026J/K move · Enter open · S save

GeoFace: Consistent Multi-View Face Generation with Geometry-Constrained Diffusion

Yeji Choi, Jinhyeok Choi, Jaewon Min, Minkyung Kwon+2

KAIST AI

Multi-View Consistency 3D Face Reconstruction Diffusion Models

Open paper AskPDF arXiv

We present GeoFace, a geometry-constrained multi-view diffusion framework for consistent face generation from a single input. % While recent multi-view diffusion models achieve photorealistic synthesis at the per-view level, they lack an explicit mechanism to enforce a shared 3D structure across views, often leading to inconsistent geometry across viewpoints. To address this, GeoFace proposes a unified dual-stream framework for joint generation of multi-view RGB images and 3D face geometry, where the appearance and geometry streams interact through shared attention layers. To encourage the two streams to mutually constrain each other, we introduce a geometry-guided attention alignment loss that supervises the cross-attention between appearance and geometry tokens with 3D-consistent correspondences, enabling the appearance stream to correctly reference pose-invariant geometric cues for robust alignment across viewpoints. Geometry is represented as a canonical UV position map, derived from a FLAME mesh fitted to multi-view observations, serving as a view-invariant shared constraint across all generated views. Experiments on RenderMe-360 and NeRSemble demonstrate that GeoFace consistently outperforms existing methods in both visual quality and cross-view geometric consistency, facilitating more efficient 3D reconstruction.

Abstract

Multi-View Image Synthesis

Explore similar work

GeoFace: Consistent Multi-View Face Generation with Geometry-Constrained Diffusion

VGGT-Diff: Visual Geometry Meets Diffusion for Sparse-View Novel View Synthesis

Repurposing Geometric Foundation Models for Multi-view Diffusion