Bridging Rendering and Generative Modeling with Monte Carlo Transport Scheduling

Junwei Shu; Wenjie Liu; Hantang Liu; Changbo Wang; Yang Li

Figure 1. Monte Carlo Transport Scheduling links rendering and generation through a shared noisy-to-clean transport view. Left: progressive path tracing, diffusion denoising, and Rectified Flow sampling each trace a trajectory from uncertainty to structure. Right: path tracing additionally provides real nested Monte Carlo endpoints along this process; we use these endpoints to define a continuous render-state space and learn a transport field toward the clean-limit image. Teaser visualization of a Monte Carlo render-state space. A low-sample input is connected to generated intermediate and clean image states by abstract trajectories representing real Monte Carlo refinement and the learned transport rollout.

Figure 2. Monte Carlo Transport Scheduling. Top: a renderer produces a nested SPP ladder of real endpoint states, forming a natural refinement trajectory. Variance time

\tau(n)

, derived from the

1/\sqrt{n}

convergence law, maps each SPP endpoint into a continuous solver clock spanning

\tau=0

(noisy) to

\tau=1

(clean). Bottom: the transport schedule is trained on real endpoints only, with two paired parameterizations—MC velocity for flow-based solvers and MC noise for diffusion-based denoisers—yielding a field that supports arbitrary-step rollout without retraining. Overview of the Monte Carlo Transport Scheduling framework.

Figure 3. Generative trajectory injection. Same-seed samples from frozen pretrained backbones are paired with samples from the same backbones when the MC transport adapter is applied after

t=0.5

. Colored boxes highlight representative regions affected by the transport bias. Notably, upon introducing physical prior biases, the resulting images display structured details instead of uniform noise, clearly capturing elements like water ripples, church architecture, and bedding textures. Comparison of frozen generative samples with and without MC transport-adapter trajectory injection for diffusion and Rectified Flow backbones.

Model	CLIPIQA $\uparrow$	MUSIQ $\uparrow$	MANIQA $\uparrow$	TOPIQ-NR $\uparrow$
RF Frozen	0.266	51.222	0.376	0.500
RF + transport adapter	0.324 (+21.5%)	51.588 (+0.7%)	0.386 (+2.9%)	0.492 (-1.5%)
DDPM Frozen	0.493	52.204	0.430	0.532
DDPM + transport adapter	0.539 (+9.3%)	57.184 (+9.5%)	0.520 (+21.1%)	0.592 (+11.4%)

Table 1. Generative trajectory injection into frozen RF and DDPM samplers. The adapter is trained on 128 samples and injected as a small residual correction with

\eta=0.05

, leaving backbone weights fixed. Parenthesized values are relative changes from the frozen backbone.

Figure 4. Stratified qualitative comparison on held-out scenes. Columns cover one sample from each of the 12 evaluation strata

\{\mathrm{L0,L1,L2,L3}\}\times\{\mathrm{enclosed,env,open}\}

. Rows show the 1 spp input, clean reference, and the four solver parameterizations used in Table 3 . All learned methods use G-buffer input and NFE 9. Samples are selected from the held-out split to cover the transport-regime/context strata and expose visible method differences. The montage illustrates both the benchmark’s transport-regime coverage and the structural differences that are under-penalized by preview-space scalar metrics. A montage with one held-out scene from each rendering-regime and context stratum, comparing input, reference, Vanilla RF, local velocity, MC velocity, and MC noise outputs.

Stop set	Target	Cases	PSNR to real target $\uparrow$
			Hold	Local Step	MC noise
Adjacent $1{\to}2,\ldots,9{\to}10$	$I_{2},\ldots,I_{10}$	90	22.160	21.006	23.084
Direct $1$ – $8{\to}10$ spp	$I_{10}$	80	17.410	19.074	20.809

Table 2. Continuous rendering-state bridge validation on RGB-only Monte Carlo endpoints. Values are average linear-RGB PSNR (dB) against the real nested-prefix target endpoint

I_{m}

. The adjacent test averages 9 first-10-SPP stops over 10 scenes; direct-to-10 averages 8 source endpoints over the same scenes.

Figure 5. Continuous bridge stops on RGB-only Monte Carlo states. Each column starts from an input endpoint, estimates a clean endpoint by subtracting the predicted MC noise, and takes the partial variance-time step toward the requested later SPP endpoint. The predicted state is compared with the corresponding low-SPP ground truth. The bottom row shows the signed update direction and an absolute target-difference heatmap; heatmap colors are normalized per panel to localize residual structure. Visual comparison of input images, MC noise bridge stops, low-SPP ground truth endpoints, and delta maps for adjacent and direct rendering-state transitions.

Space	Method	Input	NFE1	NFE2	NFE9	NFE20	NFE50	AUC logN	TV logN
HDR	Vanilla RF	RGB	20.280	20.395	19.855	19.583	19.398	19.931	0.284
	local velocity	RGB	13.581	17.098	20.099	19.082	16.552	18.040	2.573
	MC velocity	RGB	19.560	19.594	19.179	19.060	18.994	19.282	0.162
	MC noise	RGB	21.008	20.641	19.841	19.640	19.507	20.086	0.384
	Vanilla RF	GBuf	20.878	20.990	20.404	20.124	19.939	20.495	0.297
	local velocity	GBuf	13.612	17.254	20.866	19.250	15.449	18.220	3.239

Table 3. Solver formulation comparison in linear HDR and RGB preview space. The Space column marks linear HDR PSNR (HDR) or tone-mapped RGB preview PSNR (Preview). GBuf denotes RGB input augmented with albedo and normal channels. MC noise denotes the raw Monte Carlo noise parameterization evaluated from the 60k checkpoint. NFE is the number of model calls per image. AUC logN summarizes quality over the log-NFE budget range; TV logN is the normalized total PSNR variation over the same range. Bold and underline denote best and second-best within each metric/input block.

Figure 6. Linear HDR MAE across solver steps for G-buffer and RGB inputs. Unlike PSNR, which compresses rare extreme radiance failures through a logarithmic score, HDR MAE directly exposes large linear-radiance outliers. Vanilla RF develops large positive radiance outliers as NFE increases, while MC noise gives the lowest HDR error across most solver steps. OIDN and OptiX are G-buffer-input external denoisers and are shown as horizontal references only in the G-buffer panel. Insets zoom the low-error range. Two side-by-side line plots of linear HDR mean absolute error versus number of function evaluations for G-buffer and RGB inputs. Vanilla RF rises sharply with NFE, while MC noise and MC velocity remain in a low-error range; OIDN and OptiX appear as horizontal references in the G-buffer panel.

Figure 7. RGB-input qualitative comparison at NFE 1 across the 12 held-out rendering strata. Even with a single model call, MC noise preserves scene structure while maintaining strong visual quality. A qualitative montage over 12 rendering buckets comparing the noisy input, clean reference, Vanilla RF, MC velocity, and MC noise with RGB input at NFE 1.

Figure 8. Bucket-wise linear HDR PSNR at NFE 1 on the full held-out evaluation. Solver rows follow Table 3 ; OIDN and OptiX are included as G-buffer-input external denoiser references. Columns cover the 12 balanced rendering strata, and each cell reports the mean PSNR over 100 held-out samples from that bucket. Two heatmaps showing bucket-wise linear HDR PSNR for G-buffer and RGB inputs. The G-buffer panel includes Vanilla RF, OIDN, OptiX, local velocity, MC velocity, and MC noise; the RGB panel includes the four solver methods. Columns are the 12 rendering buckets formed by L0 through L3 regimes and enclosed, environment, and open transport contexts.

Bridging Rendering and Generative Modeling with Monte Carlo Transport Scheduling

Abstract

Figures & tables

Generative Modeling

Physically Based Rendering in the Latent Space

ART-MPG: Autoregressive R/T Manifold Path Guiding

One step further with Monte-Carlo sampler to guide diffusion better

Abstract

Generative Modeling

Explore similar work

Physically Based Rendering in the Latent Space

ART-MPG: Autoregressive R/T Manifold Path Guiding

One step further with Monte-Carlo sampler to guide diffusion better