Organizations: Ningbo Key Laboratory of Spatial Intelligence and Digital Derivative, Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo, Zhejiang 315200, P.R. China · Beijing Institute of Technology, Beijing, P.R. China · Shaanxi Normal University
Disentangled real-image manipulation requires factor-specific semantic control while preserving image fidelity and non-target content. StyleSpace S provides interpretable controls but is limited by GAN inversion, whereas diffusion models offer stronger image realization without an explicit factorized control basis. We introduce GeoNaviDiff, an unsupervised cross-space framework that transfers disentangled StyleSpace interventions to local Riemannian trajectories in diffusion h-space. GeoNaviDiff constructs reconstruction-decoupled supervision by extracting a spherical Log-map tangent from a matched StyleGAN reconstruction-edit pair and parallel-transporting it to the real-image diffusion state. A shared, factor-conditioned Navigator predicts this tangent at inference, while norm-preserving geodesic realization and source-anchored rollout stabilize the reverse process without edited references or factor-specific models. Experiments across multiple datasets show that GeoNaviDiff outperforms the compared methods in fidelity, factor selectivity, edit locality, and trajectory smoothness.
Figures & tables
Figure 1: Conceptual comparison of Euclidean, arbitrary geodesic, and GeoNaviDiff trajectories in diffusion h -space.
Figure 2: Training framework of GeoNaviDiff. (a) Unsupervised semantic discovery and disentangled control construction in S -space, followed by cross-space navigation into diffusion h -space. (b) The four-step parallel-transport procedure is used to construct the tangent navigation target.
Method
FFHQ
AFHQ-v2
Stanford Cars
FID ↓
PPL ↓
OFlow ↓
DCI ↑
FID ↓
PPL ↓
OFlow ↓
DCI ↑
FID ↓
PPL ↓
OFlow ↓
DCI ↑
β -TCVAE ( Chen et al. 2018 )
96.50 ± 2.40
128.85
0.12
0.28
72.00 ± 2.17
132.69
0.13
0.32
92.78 ± 2.35
133.54
0.13
0.31
FactorVAE ( Kim and Mnih 2018 )
104.80 ± 2.88
118.37
0.11
0.36
84.50 ± 2.60
122.56
0.14
0.35
115.48 ± 3.52
120.78
0.12
0.40
e4e ( Tov et al. 2021 )
75.74 ± 1.49
130.45
0.30
0.36
26.65 ± 0.55
142.69
0.23
0.31
84.08 ± 1.93
147.28
0.34
0.42
SFE ( Bobkov et al. 2024 )
76.42 ± 1.35
114.88
0.17
0.54
31.84 ± 1.12
125.71
0.19
0.49
79.21 ± 1.70
122.75
0.27
0.47
NaviGAN ( Cherepkov et al. 2021 )
76.31 ± 1.24
94.33
0.10
0.60
49.36 ± 1.09
88.95
0.12
0.57
53.60 ± 0.92
93.57
0.13
0.62
Table 1: Comparison with existing methods across VAE-, GAN-, and diffusion-based generative models. † denotes our baseline, bold indicates the best result, and “–” denotes not measured. Asyrp lacks fixed latent codes for PPL and DCI.
Figure 3: Qualitative comparison on smile manipulation across traversal levels.
Figure 4: Qualitative traversals of GeoNaviDiff across datasets and semantic factors.
Figure 5: Stepwise ablation of cross-space navigation, Riemannian realization, and source-anchored rollout.
Despite the generative capabilities of diffusion and flow models, real-image editing remains constrained by a persistent trade-off between semantic editability and structural fidelity. We trace a primary cause of this limitation to the implicit coupling of edit progress with model scale in existing paradigms. Under this coupling, stronger edits typically require visiting noisier states, which spends computation on destabilizing layout before the semantic change is well localized. We introduce NaviEdit, a training-free inference-time controller that decouples edit progress from model scale traversal through a strict self-consistency contract. NaviEdit operates at the rollout level and leaves the underlying pretrained model unchanged. It treats scale as a control input and reallocates a fixed step budget toward semantically responsive intermediate scales instead of destructive high-noise regimes. Experiments show positive average gains across compatible editors and flow backbones, supporting decoupling as a portable inference-time control principle.
Liangsi Lu, Minzhe Guo, Xuhang Chen +1
Guangdong University of Technology, Guangzhou, China · Huizhou University, Huizhou, China.
Diffusion models are a leading paradigm for data generation, but training-free editing typically re-runs the full denoising trajectory for every edit strength, making iterative refinement expensive. To address this issue, we instead edit near the data manifold, where small local updates can replace repeated re-synthesis. To enable this, we estimate a local manifold tangent space directly from perturbed samples and prove that this sample-based estimator closely approximates the true tangent. Building on this guarantee, we devise a Jacobian-free algorithm that constructs a tangent frame via small perturbations to the initial noise and alternates small tangent moves with diffusion-based projections. Updates within this frame follow principled on-manifold directions while suppressing off-manifold drift, enabling fine-grained edits without full re-diffusion or additional training. Edit strength is controlled by the number of steps for rapid, continuous adjustments that preserve fidelity and plug into existing samplers. Empirically, the resulting tangent directions yield smooth, semantic unsupervised traversals and effective CLIP-guided optimization, demonstrating practical interactive continuous editing.
Yiming Zhang, Sitong Liu, Ke Li +3
University of California San Diego, La Jolla, CA, USA · University of Washington, Seattle, WA, USA · Xidian University, Xi’an, China
Image stylization with latent-diffusion models entangles two independently refined axes: what a region depicts and how it is depicted. Such pipelines expose only global controls, yet professional retouching demands deliberate, region-specific control. We lift two conditioning weights already present in a ControlNet + IP-Adapter stylization pipeline from global scalars to per-location spatial maps, yielding local, per-axis control of content and style in a single generative pass. Because the two weights act on disjoint pathways, adjusting them independently spans a 2x2 retouching vocabulary, from free regeneration to identity preservation. We validate that edits stay confined to the retouched region and that each weight predominantly steers its own axis. Our approach requires no retraining and drops unchanged into any such pipeline.