Beyond Geometry: Artistic Disparity Synthesis for Immersive 2D-to-3D

Ping Chen; Zezhou Chen; Xingpeng Zhang; Yanlin Qian; Huan Hu; Xiang Liu; Zipeng Wang; Xin Wang

cs.CVMar 6, 2026

Beyond Geometry: Artistic Disparity Synthesis for Immersive 2D-to-3D

Authors: Ping Chen, Zezhou Chen, Xingpeng Zhang, Yanlin Qian, Huan Hu, Xiang Liu, Zipeng Wang, Xin Wang,

Organizations: Data Science & Artificial Intelligence Research Institute, China Unicom · Unicom Data Intelligence, China Unicom · School of Computer Science and Software Engineering, Southwest Petroleum University · DJI Technology Co.,Ltd. ·

Abstract

Current 2D-to-3D conversion methods achieve geometric accuracy but are artistically deficient, failing to replicate the immersive and emotionally resonant experience of professional 3D cinema. This is because geometric reconstruction paradigms mistake deliberate artistic intent, such as strategic zero-plane shifts for pop-out effects and local depth sculpting, for data noise or ambiguity. This paper argues for a new paradigm: Artistic Disparity Synthesis, shifting the goal from physically accurate disparity estimation to artistically coherent disparity synthesis. We propose Art3D, a preliminary framework exploring this paradigm. Art3D uses a dual-path architecture to decouple global depth parameters (macro-intent) from local artistic effects (visual brushstrokes) and learns from professional 3D film data via indirect supervision. We also introduce a preliminary evaluation method to quantify cinematic alignment. Experiments show our approach demonstrates potential in replicating key local out-of-screen effects and aligning with the global depth styles of cinematic 3D content, laying the groundwork for a new class of artistically-driven conversion tools.

Figures & tables

Figure 1: Artistic ambiguity in 3D film production. The same scene can exhibit distinct 3D perceptions under different artistic directions. To illustrate this, we simulate various 3D production choices and visualize their effects. From left to right, the three columns show the left view, right view, and the resulting red–cyan anaglyph (best viewed when zoomed in). Comparing the first and second rows, variations in the stereo camera’s baseline $b$ and focal length $f$ lead to different disparities, reflecting differences in the Mastery of Global Depth . The second and third rows show zero-plane ambiguity , where the perceived depth reference shifts from the foreground cow to the distant forest, often due to creative camera setup or post-production adjustment. The fourth row illustrates over-simple depth layering , where disparity remains nearly uniform, likely reflecting cost-efficient rather than artistic intent.

Figure 2: Art3D Framework. Our pipeline takes a 2D Left View as input. 1. Geometric Feature Extraction (Frozen): ‘DepthNet’ extracts geometric features and the inverse depth map, ‘geometric canvas ( $iz$ )’. 2. Artistic Blueprint & Masks (Frozen, Data Construction Stage): During the data construction stage (no training or inference required), ‘StereoNet’ estimates the target artistic blueprint ( $d^{L}$ ) and provides a valid pixel mask ( $M_{valid}$ ) via its left-right consistency check. Simultaneously, ‘Lang-SAM’ analyzes the Left View to generate the local effects mask ( $M_{local}$ ). The final global style mask ( $M_{global}$ ) is then derived from valid regions that are not part of the local effects (i.e., $M_{valid}\cdot(1-M_{local})$ ). 3. CameraNet (Trainable): This network takes features and $iz$ to synthesize the virtual camera parameters ( $vs,vt$ ), which are pixel-level tensors , and a preliminary virtual right disparity map ( $\hat{d}^{R}$ ). The virtual left disparity map ( $\hat{d}^{L}$ ) is then constructed as $iz\times vs+vt$ . 4. Dual-Path Supervision: Our core ‘ $\mathcal{L}_{Art}$ ’ loss guides the learning of $vs,vt$ by comparing $\hat{d}^{L}$ with $d^{L}$ , weighted by $M_{global}$ (for global style) and $M_{local}$ (for local effects). An additional left-right consistency check loss further refines $\hat{d}^{R}$ . 5. Virtual Right View Synthesis: The final $\hat{d}^{R}$ , combined with the original Left View , can be used to generate the Virtual Right View via standard warping techniques. For more details on the loss functions, please refer to Sec. 3 .

Operation	Details	Description
Input	Features and inverse depth ( $iz$ )	Input size = $512\times 512$
Downsampling (x3)	Conv1x1 + ReLU + Conv3x3 + ReLU	Output size = $64\times 64$ (encoder features)
Upsampling (x3)	Conv1x1 + ReLU + Deconv3x3 + ReLU	Output size = $512\times 512$ (decoder features)
Output	Conv1x1 (out_channels = 3)	Virtual camera parameters $vs$ and $vt$ , and right disparity map $\hat{d}^{R}$

Table 1: Overview of the CameraNet Architecture. This is the only component trained in our framework.

Figure 3: Qualitative analysis of Sculpting of Local Effects on 2D inputs. Row 1: Input image. Row 2: Ours (w/o $\mathcal{L}{path}(M_{local})$ ), which fails to sculpt local pop-out effects. Row 3: Owl3D, which produces partial effects but lacks artistic consistency. Row 4: Art3D (Full Model) , which successfully sculpts strong and coherent pop-out effects. Best viewed zoomed in and with red–cyan anaglyph filters.

Figure 4: Analysis of Geometric Consistency via DDC-IoU. Each row presents three independent samples. For each sample, the Anaglyph 3D view is shown on top. Below it, the corresponding right disparity map ( $d^{R}$ or $\hat{d}^{R}$ ) is displayed alongside the right geometric canvas ( $iz_{R}$ ) derived from that disparity: for ground truth, $iz_{R}$ is obtained by feeding the true right image into DepthNet; for our estimate, $iz_{R}$ is computed from the synthesized right view generated by warping the left image using $\hat{d}^{R}$ . Top Row (Original 3D Films): Visualizes the inconsistent quality of source data. The leftmost sample shows poor structural alignment ( $\text{DDC-IoU}=0$ ). The middle and rightmost samples are acceptable ( $\text{DDC-IoU}\geq 0.8$ ). Bottom Row (Our Art3D Output): Shows our model’s synthesized blueprints ( $\hat{d}^{R}$ ) for the same three scenes shown above. Our outputs consistently achieve high DDC-IoU scores (e.g., 0.85, 0.83, 0.89), demonstrating that artistic style is learned without corrupting the underlying geometry. Visually, the synthesized disparity maps are coherent and structurally sound.

Method	Immersion $\uparrow$	Visual Comfort $\uparrow$	Style Consistency $\uparrow$	Overall Preference $\uparrow$
Baseline (DA-V2)	35.2%	41.5%	22.8%	20.0%
Art3D (Ours)	64.8%	58.5%	77.2%	80.0%

Table 5: User Study : Art3D vs. Depth-Anything-V2 (Geometric-only,

N=50

videos, 25 participants)

Image Generation

Image generation studies models that create or edit images from text, examples, layouts, and other structured controls.

See all 463 papers

ArtifactWorld: Scaling 3D Gaussian Splatting Artifact Restoration via Video Generation Models
Apr 14, 2026Xinliang Wang, Yifeng Shi, Zhenyu Wu3D Gaussian Splatting Novel View Synthesis
Ke Holdings Inc. Beijing, China
ESC: Escaping 2D Diffusion Ambiguity with Geometric Constraints for Consistent Novel View Synthesis
Oct 14, 2026Ge Ya Luo, Daniel Severo, Brian Karrer +3Novel View Synthesis Multi-View Image Synthesis
FAIR at Meta · Mila – Québec AI Institute · Université de Montréal +3
ArtiFixer: Enhancing and Extending 3D Reconstruction with Auto-Regressive Diffusion Models
Feb 28, 2026Riccardo de Lutio, Tobias Fischer, Yen-Yu Chang +73D Reconstruction Novel View Synthesis
NVIDIA, USA · NVIDIA, Switzerland · ETHZ, Switzerland +4

Apr 14, 2026J/K move · Enter open · S save

ArtifactWorld: Scaling 3D Gaussian Splatting Artifact Restoration via Video Generation Models

Xinliang Wang, Yifeng Shi, Zhenyu Wu

Ke Holdings Inc. Beijing, China

3D Gaussian Splatting Novel View Synthesis Sparse-View 3D Reconstruction

Open paper AskPDF arXiv

3D Gaussian Splatting (3DGS) delivers high-fidelity real-time rendering but suffers from geometric and photometric degradations under sparse-view constraints. Current generative restoration approaches are often limited by insufficient temporal coherence, a lack of explicit spatial constraints, and a lack of large-scale training data, resulting in multi-view inconsistencies, erroneous geometric hallucinations, and limited generalization to diverse real-world artifact distributions. In this paper, we present ArtifactWorld, a framework that resolves 3DGS artifact repair through systematic data expansion and a homogeneous dual-model paradigm. To address the data bottleneck, we establish a fine-grained phenomenological taxonomy of 3DGS artifacts and construct a comprehensive training set of 107.5K diverse paired video clips to enhance model robustness. Architecturally, we unify the restoration process within a video diffusion backbone, utilizing an isomorphic predictor to localize structural defects via an artifact heatmap. This heatmap then guides the restoration through an Artifact-Aware Triplet Fusion mechanism, enabling precise, intensity-guided spatio-temporal repair within native self-attention. Extensive experiments demonstrate that ArtifactWorld achieves state-of-the-art performance in sparse novel view synthesis and robust 3D reconstruction. Code available at: https://github.com/fyting/ArtifactWorld.

Abstract

Image Generation

Explore similar work

ArtifactWorld: Scaling 3D Gaussian Splatting Artifact Restoration via Video Generation Models

ESC: Escaping 2D Diffusion Ambiguity with Geometric Constraints for Consistent Novel View Synthesis

ArtiFixer: Enhancing and Extending 3D Reconstruction with Auto-Regressive Diffusion Models