Game engines and coding agents can specify the layout, motion, and camera of a scene, yet turning such coarse structure into realistic video requires a renderer that follows the structure and completes the appearance it leaves open. Existing renderers are tied to the representation of one engine, need tens of denoising steps or a costly distillation of their own, and learn from conditions that are not aligned with their target videos. We present SolarRenderer, a structure-conditioned video renderer that adapts a frozen MiniMax-H3 33B backbone with two lightweight LoRAs. Structural videos enter the native reference pathway of the backbone at the positions of the frames they describe, without a type identifier and described only in plain text, so that a single control LoRA follows arbitrary structural representations. Starting from a pretrained four-step LoRA, structure-guided distribution matching renders in four steps, 12.3 times faster than 50-step sampling, while preserving the learned control. A block-causal variant extends the same design to streaming and renders 768p video in real time. To train the model, we build a frame-aligned corpus of 136K clips from game engines, simulators, and YouTube videos. Experiments on an in-distribution test set and a generalization benchmark of 16 structural representations show that SolarRenderer achieves state-of-the-art results across both, and generalizes to 11 unseen representations, including pose skeletons, drawings, and point clouds.
Figures & tables
Figure 1: \texorpdfstring \pdfxform D SolarRenderer reads any structural video through the native reference pathway of a frozen DiT. The sequence of the single-stream DiT holds text, the optional first frame I , structural latents sk , and noisy latents zk , each sk taking the RoPE position (t,h,w) of zk and I index 0. 1 The control LoRA ϕ is trained by flow matching. 2 The acceleration LoRA ψ is distilled toward the teacher with S , extrapolated by w from the one without S .
Figure 2: The training corpus combines 136K clips from eight sources. Each sector of the ring shows one source, with the condition on the inner ring (the first frame for WildWorld and OmniWorld) and the ground-truth frame of the same clip on the outer ring. The panels give the scale and construction steps of the data, the clips of each source by input combination, the time of day and scenes named in the captions, and the most frequent words of the captions across all training clips.
Table 1: Comparison with existing methods. Gemini Elo ratings for the four judge questions and the automatic score (Auto.) on both test sets. The causal variant is rated only against ours. Methods that lose every comparison on a question share one rating. Bold and underlined italics mark the best and second-best values.