Participating media are a pervasive and intriguing visual effect in virtual environments. Unfortunately, rendering such phenomena in real-time is notoriously difficult due to the computational expense of estimating the volume rendering equation. While the six-way lightmaps technique has been widely used in video games to render smoke with a camera-oriented billboard and approximate lighting effects using six precomputed lightmaps, achieving a balance between realism and efficiency, it is limited to pre-simulated animation sequences and is ignorant of camera movement. In this work, we propose a neural six-way lightmaps method to strike a long-sought balance between dynamics and visual realism. Our approach first generates a guiding map from the camera view using ray marching with a large sampling distance to approximate smoke scattering and silhouette. Then, given a guiding map, we train a neural network to predict the corresponding six-way lightmaps. The resulting lightmaps can be seamlessly used in existing game engine pipelines. This approach supports visually appealing rendering effects while enabling real-time user interactivity, including smoke-obstacle interaction, camera movement, and light change. By conducting a series of comprehensive benchmarks, we demonstrate that our method is well-suited for real-time applications, such as games and VR/AR.
Figures & tables
Figure 1. Two frames of a chimney smoke example rendered with rotating camera direction in Unreal Engine (UE) ( Epic Games, 2021 ) . Our neural lightmaps support dynamic lighting, camera direction, and realistic multiple-scattering effects, whereas traditional six-way lightmaps ( Muller, 2023 ) with asymmetrically shaped smoke only work for one camera direction, leading to severe offset artifacts and a lack of interaction. Compared with UE built-in Niagara system using sprite-based particle effects, our method produces substantially more realistic smoke appearance.
Figure 2. An example set of lightmaps packed in two RGBA textures. Of the 8 available channels, 6 channels store the six-way scattering lightmaps along axis-aligned directions. Additionally, the alpha channel for the first texture contains the transparency T(x↔z) , while the alpha channel of the second texture is an optional emissive component.
Method
Dyn. light
Dyn. smoke
Dyn. view
Obstacle
Speed
Traditional
Yes
No
No ⋆
No
Very fast
Ours neural
Yes
Yes
Yes
Yes
Fast
Table 1. Comparison between traditional and our neural six-way lightmaps. Dyn. denotes dynamic. (No ⋆ : Traditional method renders a billboard under any view, which introduces large artifact as shown in Fig. 1 )
Figure 3. Our pipeline: the physically based fluid simulator takes the obstacle as input (a) to produce the density field (b). (c) A ray marching with a large sample step extracts the guiding map with three channels, in-scattered radiance L~scattering , transparency T , and depth D . (d) Our neural lightmaps generator contains a modified UNet that first extracts channel-shared features from the input, which are then processed by four dedicated channel adapters with outputs for front and back, left and right, up and down, and transparency and emissive, respectively. The shadow map of obstacle (e) is generated from the light and combined with predicted lightmaps to generate the final rendering result (f).
Figure 4. Our approach can handle dynamic smoke under a moving camera and be integrated seamlessly into Unreal Engine ( Epic Games, 2021 ) .