Monte Carlo rendering and modern generative models both transform uncertain states into structured images, yet they are usually studied as separate processes. We introduce Monte Carlo Transport Scheduling, a framework that treats progressive path tracing as a continuous sampling-driven transport process. Our key observation is that the renderer already produces physically valid states along this process: nested Monte Carlo estimates trace a refinement trajectory whose natural time coordinate follows from sampling variance. This view leads to a continuous training framework that learns from real render endpoints rather than synthetic interpolants, preserving the statistical structure of Monte Carlo estimation while enabling arbitrary-step neural refinement. We evaluate the framework on a controlled rendering benchmark designed to separate transport difficulty from scene context, and show that it yields stable render refinement, supports continuous stopping between rendering states, and transfers as a physical prior for frozen generative samplers. These results suggest a common continuous-time substrate for rendering and generation, where Monte Carlo sampling provides both the physical states and the supervision for learning image transport.
Figures & tables
Figure 1. Monte Carlo Transport Scheduling links rendering and generation through a shared noisy-to-clean transport view. Left: progressive path tracing, diffusion denoising, and Rectified Flow sampling each trace a trajectory from uncertainty to structure. Right: path tracing additionally provides real nested Monte Carlo endpoints along this process; we use these endpoints to define a continuous render-state space and learn a transport field toward the clean-limit image. Teaser visualization of a Monte Carlo render-state space. A low-sample input is connected to generated intermediate and clean image states by abstract trajectories representing real Monte Carlo refinement and the learned transport rollout.
Figure 2. Monte Carlo Transport Scheduling. Top: a renderer produces a nested SPP ladder of real endpoint states, forming a natural refinement trajectory. Variance time τ(n) , derived from the 1/n convergence law, maps each SPP endpoint into a continuous solver clock spanning τ=0 (noisy) to τ=1 (clean). Bottom: the transport schedule is trained on real endpoints only, with two paired parameterizations—MC velocity for flow-based solvers and MC noise for diffusion-based denoisers—yielding a field that supports arbitrary-step rollout without retraining. Overview of the Monte Carlo Transport Scheduling framework.
Figure 3. Generative trajectory injection. Same-seed samples from frozen pretrained backbones are paired with samples from the same backbones when the MC transport adapter is applied after t=0.5 . Colored boxes highlight representative regions affected by the transport bias. Notably, upon introducing physical prior biases, the resulting images display structured details instead of uniform noise, clearly capturing elements like water ripples, church architecture, and bedding textures. Comparison of frozen generative samples with and without MC transport-adapter trajectory injection for diffusion and Rectified Flow backbones.
Model
CLIPIQA ↑
MUSIQ ↑
MANIQA ↑
TOPIQ-NR ↑
RF Frozen
0.266
51.222
0.376
0.500
RF + transport adapter
0.324 (+21.5%)
51.588 (+0.7%)
0.386 (+2.9%)
0.492 (-1.5%)
DDPM Frozen
0.493
52.204
0.430
0.532
DDPM + transport adapter
0.539 (+9.3%)
57.184 (+9.5%)
0.520 (+21.1%)
0.592 (+11.4%)
Table 1. Generative trajectory injection into frozen RF and DDPM samplers. The adapter is trained on 128 samples and injected as a small residual correction with η=0.05 , leaving backbone weights fixed. Parenthesized values are relative changes from the frozen backbone.
Figure 4. Stratified qualitative comparison on held-out scenes. Columns cover one sample from each of the 12 evaluation strata {L0,L1,L2,L3}×{enclosed,env,open} . Rows show the 1 spp input, clean reference, and the four solver parameterizations used in Table 3 . All learned methods use G-buffer input and NFE 9. Samples are selected from the held-out split to cover the transport-regime/context strata and expose visible method differences. The montage illustrates both the benchmark’s transport-regime coverage and the structural differences that are under-penalized by preview-space scalar metrics. A montage with one held-out scene from each rendering-regime and context stratum, comparing input, reference, Vanilla RF, local velocity, MC velocity, and MC noise outputs.
Stop set
Target
Cases
PSNR to real target ↑
Hold
Local Step
MC noise
Adjacent 1→2,…,9→10
I2,…,I10
90
22.160
21.006
23.084
Direct 1 – 8→10 spp
I10
80
17.410
19.074
20.809
Table 2. Continuous rendering-state bridge validation on RGB-only Monte Carlo endpoints. Values are average linear-RGB PSNR (dB) against the real nested-prefix target endpoint Im . The adjacent test averages 9 first-10-SPP stops over 10 scenes; direct-to-10 averages 8 source endpoints over the same scenes.
Figure 5. Continuous bridge stops on RGB-only Monte Carlo states. Each column starts from an input endpoint, estimates a clean endpoint by subtracting the predicted MC noise, and takes the partial variance-time step toward the requested later SPP endpoint. The predicted state is compared with the corresponding low-SPP ground truth. The bottom row shows the signed update direction and an absolute target-difference heatmap; heatmap colors are normalized per panel to localize residual structure. Visual comparison of input images, MC noise bridge stops, low-SPP ground truth endpoints, and delta maps for adjacent and direct rendering-state transitions.
Space
Method
Input
NFE1
NFE2
NFE9
NFE20
NFE50
AUC logN
TV logN
HDR
Vanilla RF
RGB
20.280
20.395
19.855
19.583
19.398
19.931
0.284
local velocity
RGB
13.581
17.098
20.099
19.082
16.552
18.040
2.573
MC velocity
RGB
19.560
19.594
19.179
19.060
18.994
19.282
0.162
MC noise
RGB
21.008
20.641
19.841
19.640
19.507
20.086
0.384
Vanilla RF
GBuf
20.878
20.990
20.404
20.124
19.939
20.495
0.297
local velocity
GBuf
13.612
17.254
20.866
19.250
15.449
18.220
3.239
Table 3. Solver formulation comparison in linear HDR and RGB preview space. The Space column marks linear HDR PSNR (HDR) or tone-mapped RGB preview PSNR (Preview). GBuf denotes RGB input augmented with albedo and normal channels. MC noise denotes the raw Monte Carlo noise parameterization evaluated from the 60k checkpoint. NFE is the number of model calls per image. AUC logN summarizes quality over the log-NFE budget range; TV logN is the normalized total PSNR variation over the same range. Bold and underline denote best and second-best within each metric/input block.
Figure 6. Linear HDR MAE across solver steps for G-buffer and RGB inputs. Unlike PSNR, which compresses rare extreme radiance failures through a logarithmic score, HDR MAE directly exposes large linear-radiance outliers. Vanilla RF develops large positive radiance outliers as NFE increases, while MC noise gives the lowest HDR error across most solver steps. OIDN and OptiX are G-buffer-input external denoisers and are shown as horizontal references only in the G-buffer panel. Insets zoom the low-error range. Two side-by-side line plots of linear HDR mean absolute error versus number of function evaluations for G-buffer and RGB inputs. Vanilla RF rises sharply with NFE, while MC noise and MC velocity remain in a low-error range; OIDN and OptiX appear as horizontal references in the G-buffer panel.
Figure 7. RGB-input qualitative comparison at NFE 1 across the 12 held-out rendering strata. Even with a single model call, MC noise preserves scene structure while maintaining strong visual quality. A qualitative montage over 12 rendering buckets comparing the noisy input, clean reference, Vanilla RF, MC velocity, and MC noise with RGB input at NFE 1.
Figure 8. Bucket-wise linear HDR PSNR at NFE 1 on the full held-out evaluation. Solver rows follow Table 3 ; OIDN and OptiX are included as G-buffer-input external denoiser references. Columns cover the 12 balanced rendering strata, and each cell reports the mean PSNR over 100 held-out samples from that bucket. Two heatmaps showing bucket-wise linear HDR PSNR for G-buffer and RGB inputs. The G-buffer panel includes Vanilla RF, OIDN, OptiX, local velocity, MC velocity, and MC noise; the RGB panel includes the four solver methods. Columns are the 12 rendering buckets formed by L0 through L3 regimes and enclosed, environment, and open transport contexts.
Image diffusion models have shown impressive image generation capabilities but are often hard to control, in contrast to classical computer graphics pipelines such as physically based rendering. However, we observe that there is a bridge between light transport phenomena and the distribution of latent space values produced by such models. Thus, we introduce physically based rendering in the feature space learned by the variational autoencoders in generative models, enabling light transport simulation in the latent space. This allows us to leverage physically based rendering techniques to output latent maps for physically guided content generation. We propose modifications to the rendering equation, which, when paired with a differentiable renderer, can yield an optimal set of scene parameters that require only minimal refinement to accurately render into the pretrained latent space. We train our method on a single rendered image, and then demonstrate the generalization of the method to scene geometry changes, lighting changes, and camera view changes.
Specular chains concentrate important transport in small regions of path space and the structure depends on chain topology. We introduce ART-MPG, a compact neural model that samples chain length, an autoregressive reflection/transmission (R/T) string, and a direction from a topology-conditioned four-component von Mises-Fisher (vMF) mixture. We extend the repeated trial weighting in Manifold Path Guiding (MPG) by conditioning retry sampling on both chain length and the sampled R/T string, accounting for its known mass outside the stochastic re-hit trials. For a fixed proposal and target chain, this preserves the ideal expectation while reducing the expected retry count and reciprocal-estimator variance compared with retries conditioned on chain length alone. We distinguish the formal unbiasedness result from the capped and clamped renderer used in experiments. We evaluate fourteen scenes across a range of samples per pixel (SPP) against existing online and offline rendering algorithms. Our method consistently produces lower mean squared error at equal SPP than the online and offline baselines, with lower standard deviation across runs.
Zehao Zhao, Yanyang Yang, Xiaole Zhang +1
Ming Hsieh Department of Electrical and Computer Engineering, University of Southern California · Computational Science and Engineering, Georgia Institute of Technology · Department of Electrical and Computer Engineering, Carnegie Mellon University
Stochastic differential equation (SDE)-based generative models have achieved substantial progress in conditional generation via training-free differentiable loss-guided approaches. However, existing methodologies utilizing posterior sam- pling typically confront a substantial estimation error, which results in inaccu- rate gradients for guidance and leading to inconsistent generation results. To mitigate this issue, we propose that performing an additional backward denois- ing step and Monte-Carlo sampling (ABMS) can achieve better guided diffu- sion, which is a plug-and-play adjustment strategy. To verify the effectiveness of our method, we provide theoretical analysis and propose the adoption of a dual-focus evaluation framework, which further serves to highlight the critical problem of cross-condition interference prevalent in existing approaches. We conduct experiments across various task settings and data types, mainly includ- ing conditional online handwritten trajectory generation, image inverse problems (inpainting, super resolution and gaussian deblurring) molecular inverse design and so on. Experimental results demonstrate that our approach can be effec- tively used with higher order samplers and consistently improves the quality of generation samples across all the different scenarios.
Minsi Ren, Wenhao Deng, Ruiqi Feng +1
AI for Scientific Simulation and Discovery Lab, Westlake University