cs.GRFeb 18, 2026
SaveStyle-Aware Gloss Control for Generative Non-Photorealistic Rendering
Organizations: University of Zaragoza – I3A, Spain
Abstract
Humans can infer material characteristics of objects from their visual appearance, and this ability extends to artistic depictions, where similar perceptual strategies guide the interpretation of paintings or drawings. Among the factors that define material appearance, gloss, along with color, is widely regarded as one of the most important, and recent studies indicate that humans can perceive gloss independently of the artistic style used to depict an object. To investigate how gloss and artistic style are represented in learned models, we train an unsupervised generative model on a newly curated dataset of painterly objects designed to systematically vary such factors. Our analysis reveals a hierarchical latent space in which gloss is disentangled from other appearance factors, allowing for a detailed study of how gloss is represented and varies across artistic styles. Building on this representation, we introduce a lightweight adapter that connects our style- and gloss-aware latent space to a latent-diffusion model, enabling the synthesis of non-photorealistic images with fine-grained control of these factors. We compare our approach with previous models and observe improved disentanglement and controllability of the learned factors.
Figures & tables
Figure 1: Left: We train a pSp–StyleGAN2 pipeline that learns a 16-layer latent space where appearance factors such as gloss and artistic style emerge in a disentangled and hierarchical manner. Specific layers specialize in different attributes, e.g., Layer 6 captures gloss and Layer 8 captures style. Right: This learned space enables intuitive control of appearance. Given a reference geometry (edges) and optional albedo map, the user can employ our diffusion-based pipeline to transfer the gloss and style of an input drawing (inset) to new objects, or traverse the gloss dimension to obtain predictable variations from matte to glossy while keeping other factors stable.
Figure 2: Example of brushstroke map extraction for a matte sphere painted in charcoal style. Artist-painted reference ( left ), corresponding photorealistic rendering ( middle ), and the estimated brushstroke map extracted from the pair ( right ).
Figure 3: Application of the brushstroke map s to generate controlled style-guidance samples. From top to bottom rows, we show 1) the photorealistic reference spheres with gradually varying roughness; 2) the stylized spheres using the estimated brushstroke map s ; 3) the result of applying StyLit to the reference geometry g ; and 4) the corresponding samples from the original Subias et al. dataset [ 38 ] . Note the more controlled and continuous increment of gloss of our samples when compared with the reference.
Figure 4: Diagram of the architecture composed by the pSp encoder that constructs a layer-wise latent space through map2style layers, and the StyleGAN2 generator, which synthesizes images conditioned on these layer-wise latent representations.
Figure 5: Traversals at different depths of the latent space produce interpretable changes in appearance. Starting from the embeddings of a source image ( left ), moving along early ( top ), intermediate ( middle ), and late ( bottom ) layers induces variations in geometry and illumination, gloss and style, or color, respectively.
Figure 6: Per-layer corrected conditional mutual information scores between generative factors and layers, illustrating how different attributes are encoded across the latent hierarchy after accounting for the influence of the remaining factors.
Figure 7: Projection of samples onto a one-dimensional space (horizontal axis) learned with a linear ridge regression model. For all samples at each true gloss level (vertical axis), we show their scattering in the projected space, with small vertical jitter for visibility, together with the resulting half violin plot. Each point is projected using a model that was trained without that sample (cross-validation).
Figure 8: Representative 2D t-SNE visualizations illustrating the internal separation of samples within each label. Top: Early layers colored by geometry labels. Bottom: Late layers colored by color labels. Right: Layer 8 colored by style layers.
Figure 9: Uniqueness of gloss information per layer, measured by variations when predicting gloss labels without layer . Please refer to the text for more details.
| #Styles | 2 | 3 | 4 | 5 |
|---|---|---|---|---|
| Reg ↓ | 11.13 | 18.40 | 21.68 | 22.13 |
| PPL ↓ | 54.32 | 54.03 | 60.22 | 55.20 |
Table 1: Quantitative results for the ablation study on the number of styles. Reported metrics represent compactness ( Reg , lower is better), and smoothness ( PPL , lower is better) of the latent spaces.
Figure 10: Diagram of the proposed diffusion-based pipeline. It can handle up to three inputs: (i) a text prompt defining the color, geometry, and illumination, (ii) a style and gloss reference image that uses the intermediate layers of the latent space created in previous sections, and (iii) an optional image to guide spatial information of geometry and albedo.
Figure 11: Examples of three input modalities supported by our pipeline . Our method allows different levels of conditioning on geometry and color: color can be specified through a text prompt ( left ) or through spatial guidance using an albedo map ( middle ). Geometry is provided as edge or border maps, and by leveraging a ControlNet pretrained on Canny edges the pipeline also generalizes to other geometric representations such as clip-art drawings ( right ).
Figure 12: Qualitative comparison with state-of-the-art models . General-purpose T2I models produce visually striking images but often deviate from the target style and geometry. GPT Image 1.5 can often approximate the reference style, but its outputs offer only limited fine-grained control and may vary across prompts, making precise manipulation more challenging than with our method ( Fig. 13 ). Style-transfer methods better respect the reference style but tend to generate less compelling or less stable outputs. Our method achieves visually appealing results while maintaining high fidelity to both the reference geometry and the intended painterly style.
Figure 13: Comparison of gloss control . We compare our method against a diverse set of baselines: ConceptSliders [ 19 ] , a LoRA-based method; StyleID [ 34 ] , a training-free diffusion-based style-transfer method; GPT Image 1.5 [ 87 ] , a general-purpose image-generation model; and Artist-Inator [ 38 ] , the closest prior work to ours, which is based on ControlNet. For our method, the target gloss level is controlled with a slider, shown at the top of the figure. For Artist-Inator and GPT Image 1.5, it is specified through a text prompt, shown at the bottom and ablated in Sec. S8 of the supplementary material. For StyleID and ConceptSliders, gloss is varied by traversing a hyperparameter (see Sec. S7.2 for details). Our approach produces a clear, gradual modulation of gloss while preserving geometry and overall style. In contrast, the baselines have limited ability to produce smooth, continuous gloss changes while maintaining identity and fine details. GPT Image 1.5, in particular, often produces visually striking results but struggles to maintain a consistent gloss progression.
| Method | Mean Rank ↓ | RP ↓ | Pref. ↑ |
|---|---|---|---|
| Ours | 1.534 | – | |
| InstantStyle | 2.919 | 80.95% | |
| StyleID | 3.274 | 76.59% | |
| GPT Image 1.5 | 3.713 | 85.71% | |
| Artist-Inator | 4.036 | 87.70% | |
| FLUX | 4.273 | 87.30% |
Table 2: Quantitative results for Experiment 1 (Style transfer), comparing our method with alternative style transfer approaches. We report the mean rank with standard deviation, the Rank Product (RP), and the preference rate of our method over each alternative (Pref.). Lower mean rank and RP indicate better performance, while higher preference indicates better performance of our model.
Figure 14: Distribution of preference rankings assigned to each evaluated method in Experiment 1, aggregated across all participants and the 12 evaluation examples. Each stacked bar shows the total number of times a method received each ranking position (R1–R7). Pairwise statistical comparisons reveal significant differences between methods, except for those enclosed in gray boxes, which are not statistically distinguishable.
| Method | MAE ↓ | RMSE ↓ | %Exact ↑ |
|---|---|---|---|
| Ours | 0.37 | 0.79 | |
| GPT Image 1.5 | 0.58 | 0.66 | |
| ConceptSliders | 1.18 | 0.48 | |
| Artist-Inator | 1.21 | 0.43 |
Table 3: Quantitative results for Experiment 2 (Gloss ordering) across all trials. We report the mean absolute error with standard deviation, RMSE, and exact-match rate. Lower error and RMSE indicate better performance, while higher exact rate indicates better performance.
Figure 15: Bias in general-purpose models. Example illustrating how learned priors in large-scale text-to-image models can conflict with user prompts. In this case, GPT Image 1.5 struggles to generate glossy pineapples and instead introduces pineapple-like surface textures as gloss increases. In contrast, our disentangled representation enables more precise and continuous control over gloss while preserving the underlying object appearance.
More in this topic
Controllable Image Generation
Controllable image generation steers images using spatial, visual, semantic, stylistic, or physical constraints beyond ordinary text prompts.
See all 215 papers