Coarse-to-real (C2R) video generation converts coarse animation, simulation, or game-rendered videos into photorealistic videos while preserving scene layout, camera motion, and dynamics. Training such models requires aligned (coarse, realistic) video pairs, which are expensive to create and confine prior systems to narrow synthetic domains. We exploit a simple asymmetry: paired videos are hard to obtain, but paired real-to-coarse images are easy to construct with existing editing and 3D reconstruction tools. Using these image pairs, we train the DINO-R2C model to transfer realistic images to coarse images in DINO feature space, and apply it frame-wise to ordinary real videos to synthesize pseudo-coarse videos. The C2R# model then learns to generate each real video from its pseudo-coarse counterpart, free from needing (coarse, realistic) video pairs. At inference, the DINO-R2C model is discarded and actual coarse videos drive the C2R# model directly. Thus, our pair-free training scales with any real-video corpora rather than with paired synthetic videos. We will release the model and project webpage at https://gonzalognogales.github.io/coarse2real-sharp/.
Figures & tables
Figure 1: The C2R# model transforms game renders (top), geometry blockouts (middle), and stylized animations (bottom) into photorealistic videos.
Figure 2: Overview of the DINO-R2C and C2R# models. Top: The DINO-R2C model is trained from paired realistic and coarse images using feature reconstruction and pixel-space reconstruction losses. Once trained, it is applied to real videos frame-wise to synthesize pseudo-coarse videos. Bottom left: The C2R# model is trained on these pseudo-paired videos. The pseudo-coarse video is encoded by the frozen DINO encoder E and injected into the frozen Wan2.2 backbone through trainable control and fusion adapters, while the original real video provides the training target. Bottom right: at inference, an actual coarse video is encoded directly by E and used to condition the same C2R# model; the DINO-R2C model is no longer required. Orange modules are trainable, blue modules are frozen, gray boxes are tensors, and red boxes denote training losses.
Figure 3: Example (coarse, realistic) image pairs. A realistic image (left) is paired with coarse images in four modalities (right): Coarser geometry, Coarse geometry, Coarser Anime, and Coarse Anime.
DINO-R2C Recipe
Perceptual flicker (relative) ↓
Smoothed-frame flicker (relative) ↓
Motion error (EPE) ↓
Motion magnitude ratio ≈1
Boundary stability (Dice) ↑
Content consistency (DINO) ↑
None, no LPIPS
2.2028
4.1565
2.2685
2.1909
0.7195
0.9816
None + LPIPS
2.1793
3.9131
1.8678
1.8304
0.7271
0.9877
0.5× + LPIPS
0.9314
1.7901
0.8819
1.0844
0.8345
0.9946
1× , no LPIPS
0.7965
1.6825
0.9066
1.1504
0.8327
0.9914
1× + LPIPS
0.9023
1.7108
0.8695
1.0875
0.8354
0.9946
2× + LPIPS (ours)
0.8649
1.6294
0.8429
1.0797
0.8376
0.9948
Table 1: DINO-R2C design ablations, video stability. Evaluated on 1,000 real videos in the Coarse geometry modality at 256×256 . Columns report means across videos; both flicker measures are per-video medians relative to source-video error. Bold/underlined values are best/second-best. Metrics are explained in Section 4.2 and Appendix B.3 .
Figure 4: Effect of the RGB bottleneck during training. Matched frames show the control input (left), C2R# (center), and a variant conditioned directly on predicted DINO-R2C features, bypassing decoding and re-encoding (right). The latter produces visible artifacts.
Figure 5: Qualitative comparison with baseline methods. Three groups of two rows show game renders (top), geometry blockouts (middle), and stylized animations (bottom). “Rep.” denotes our Editto Sim2Real and VACE 2.2 reproductions, fine-tuned on the same pseudo-paired videos as C2R# (ours).
Figure 6: Domain transfer in CSD-L style-feature space. Left: DINO-R2C maps 1,000 real videos to four distinct pseudo-coarse style clusters. Right: C2R# maps actual coarse videos toward the real-reference distribution. Points denote videos; arrows connect input/output centroids. PCA is fitted per panel (Appendix B.4 ).
Figure 7: Limitations under abrupt control changes. Input (left) and C2R# output (right): abrupt control changes can lead to unwanted content and reduced alignment.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
DINO-R2C Recipe
Latent MSE ↓
RGB MSE ↓
RGB L1 ↓
LPIPS ↓
DINO patch cosine ↑
Depth RMSE ↓
SSIM ↑
Boundary F1 ↑
No Aug, no LPIPS
0.4443
0.0217
0.1138
0.3980
0.9055
0.4537
0.6225
0.3995
No Aug, w/ LPIPS
0.4406
0.0218
0.1137
0.3707
0.9121
0.4447
0.6122
0.5150
0.5× Aug, w/ LPIPS
0.4296
0.0214
0.1129
0.3700
0.9128
0.4380
0.6236
0.4654
1× Aug, no LPIPS
0.4276
0.0211
0.1123
0.4127
0.9020
0.4483
0.6359
0.2889
1× Aug, w/ LPIPS
0.4280
0.0212
0.1126
0.3682
0.9127
0.4357
0.6247
0.4644
2× Aug, w/ LPIPS (ours)
0.4237
0.0212
0.1122
0.3659
0.9136
0.4352
0.6282
0.4573
Appendix
Table 3: DINO-R2C design ablations, image quality. Evaluated on 1,000 held-out images in the Coarse geometry modality, comparing predictions with paired coarse targets. Latent MSE measures feature reconstruction; RGB MSE and L1 measure pixel errors; LPIPS and DINO patch cosine measure perceptual and local content agreement; depth RMSE, SSIM, and boundary F1 assess geometry, local image structure, and outlines. Latent MSE uses the native feature grid; other metrics use 256×256 images. Values are means across images. Bold and underlining mark the best and second-best values. Definitions are given in Appendix B.3 .
Figure 8: Qualitative DINO-R2C results. Each example shows a realistic image and decoded coarse predictions for Coarser geometry, Coarse geometry, Coarser Anime, and Coarse Anime. The predictions illustrate modality-dependent appearance transfer while retaining scene structure.
Method / training stage
Total params
Trainable params
Trainable (%)
C2R orig. : Stage I
14.288B
14.288B
100.00
C2R orig. : Stage II
14.333B
44.33M
0.31
Editto Global
17.460B
122.68M
0.70
Editto Sim2Real
17.460B
122.68M
0.70
VACE 2.2
34.675B
6.098B
17.59
C2R# (ours)
28.758B
180.54M
0.63
Appendix
Table 4: Training parameters. Totals include the diffusion backbone and control/fusion modules, including unmerged LoRA where applicable, but exclude frozen external text and feature encoders, the VAE, and separately trained DINO-R2C models. Counts include both denoising experts for VACE 2.2 and the C2R# . B and M denote billions and millions of parameters.
Quantity
Value
Model params
Shared backbone + four heads
319.92 M trainable
Training efficiency
Hardware
3×8 B200 GPUs
Training iterations
20,000
Global batch size
192
Appendix
Table 5: Model size and efficiency for a shared-backbone, four-head DINO-R2C architecture. Inference uses BF16 at 1280×720 and includes the projector and frozen RAE decoder, excluding model loading, transfers, video codecs, and storage I/O.
Figure 9: The C2R# model with a reference image. Each example includes a control video, the reference image used at inference, a text prompt, and the generated result. Examples cover game renders, geometry blockouts, and stylized animations. This extension uses separately trained weights and an additional image input, and is evaluated separately from the standard control-video-and-text setting.
Figure 10: First-attempt comparison with Seedance 2.5. Columns show the input control, C2R# output, and Seedance 2.5 output. Rows show examples of game renders, geometry blockouts, and stylized animations, respectively. Both methods receive the same control video and text prompt, and the displayed results are from the first generation attempt. In these examples, the C2R# model produces a more photorealistic appearance, while Seedance 2.5 retains more of the coarse input style or changes the scene layout.
Despite recent progress, video diffusion models still struggle to synthesize realistic videos involving highly dynamic motions or requiring fine-grained motion controllability. A central limitation lies in the scarcity of such examples in commonly used training datasets. To address this, we introduce DynaVid, a video synthesis framework that leverages synthetic motion data in training, which is represented as optical flow and rendered using computer graphics pipelines. This approach offers two key advantages. First, synthetic motion offers diverse motion patterns and precise control signals that are difficult to obtain from real data. Second, unlike rendered videos with artificial appearances, rendered optical flow encodes only motion and is decoupled from appearance, thereby preventing models from reproducing the unnatural look of synthetic videos. Building on this idea, DynaVid adopts a two-stage generation framework: a motion generator first synthesizes motion, and then a motion-guided video generator produces video frames conditioned on that motion. This decoupled formulation enables the model to learn dynamic motion patterns from synthetic data while preserving visual realism from real-world videos. We validate our framework on two challenging scenarios, vigorous human motion generation and extreme camera motion control, where existing datasets are particularly limited. Extensive experiments demonstrate that DynaVid improves the realism and controllability in dynamic motion generation and camera motion control.
We present Paris 2.0, the first video generation model pre-trained through decentralized computation. Its training recipe builds upon Paris 1.0 (arXiv:2510.03434), the first ever open-weight Decentralized Diffusion Model (DDM), which showed that image generation can be trained without a monolithic GPU cluster. However, temporally coherent video generation had remained an open problem under decentralized training, and Paris 2.0 closes it. In low-resolution text-to-video training, against a monolithic model trained on the same data under a matched total compute budget, Paris 2.0 cuts Frechet Video Distance (FVD) from 561.04 to 279.01, a ~2.0x improvement, and lifts CLIP text-video similarity and aesthetic score.
Video Diffusion Models (VDMs) offer a promising approach for simulating dynamic scenes and environments, with broad applications in robotics and media generation. However, existing models often generate temporally incoherent content that violates basic physical intuition, significantly limiting their practical applicability. We propose PhysAlign, an efficient framework for physics-coherent image-to-video (I2V) generation that explicitly addresses this limitation. To overcome the critical scarcity of physics-annotated videos, we first construct a fully controllable synthetic data generation pipeline based on rigid-body simulation, yielding a highly-curated dataset with accurate, fine-grained physics and 3D annotations. Leveraging this data, PhysAlign constructs a unified physical latent space by coupling explicit 3D geometry constraints with a Gram-based spatio-temporal relational alignment that extracts kinematic priors from video foundation models. Extensive experiments demonstrate that PhysAlign significantly outperforms existing VDMs on tasks requiring complex physical reasoning and temporal stability, without compromising zero-shot visual quality. PhysAlign shows the potential to bridge the gap between raw visual synthesis and rigid-body kinematics, establishing a practical paradigm for genuinely physics-grounded video generation. The project page is available at https://physalign.github.io/PhysAlign.
Zhexiao Xiong, Yizhi Song, Liu He +4
Washington University in St. Louis · Purdue University · NVIDIA