cs.GRFeb 18, 2026

Style-Aware Gloss Control for Generative Non-Photorealistic Rendering

Authors: Santiago Jimenez-Navarro, Belen Masia, Ana Serrano

Organizations: University of Zaragoza – I3A, Spain

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Humans can infer material characteristics of objects from their visual appearance, and this ability extends to artistic depictions, where similar perceptual strategies guide the interpretation of paintings or drawings. Among the factors that define material appearance, gloss, along with color, is widely regarded as one of the most important, and recent studies indicate that humans can perceive gloss independently of the artistic style used to depict an object. To investigate how gloss and artistic style are represented in learned models, we train an unsupervised generative model on a newly curated dataset of painterly objects designed to systematically vary such factors. Our analysis reveals a hierarchical latent space in which gloss is disentangled from other appearance factors, allowing for a detailed study of how gloss is represented and varies across artistic styles. Building on this representation, we introduce a lightweight adapter that connects our style- and gloss-aware latent space to a latent-diffusion model, enabling the synthesis of non-photorealistic images with fine-grained control of these factors. We compare our approach with previous models and observe improved disentanglement and controllability of the learned factors.

Figures & tables

Figure 1: Left: We train a pSp–StyleGAN2 pipeline that learns a 16-layer latent space where appearance factors such as gloss and artistic style emerge in a disentangled and hierarchical manner. Specific layers specialize in different attributes, e.g., Layer 6 captures gloss and Layer 8 captures style. Right: This learned space enables intuitive control of appearance. Given a reference geometry (edges) and optional albedo map, the user can employ our diffusion-based pipeline to transfer the gloss and style of an input drawing (inset) to new objects, or traverse the gloss dimension to obtain predictable variations from matte to glossy while keeping other factors stable.
Figure 2: Example of brushstroke map extraction for a matte sphere painted in charcoal style. Artist-painted reference ( left ), corresponding photorealistic rendering ( middle ), and the estimated brushstroke map extracted from the pair ( right ).
Figure 3: Application of the brushstroke map s to generate controlled style-guidance samples. From top to bottom rows, we show 1) the photorealistic reference spheres with gradually varying roughness; 2) the stylized spheres using the estimated brushstroke map s ; 3) the result of applying StyLit to the reference geometry g ; and 4) the corresponding samples from the original Subias et al. dataset [ 38 ] . Note the more controlled and continuous increment of gloss of our samples when compared with the reference.
Figure 4: Diagram of the architecture composed by the pSp encoder that constructs a layer-wise latent space through map2style layers, and the StyleGAN2 generator, which synthesizes images conditioned on these layer-wise latent representations.
Figure 5: Traversals at different depths of the latent space produce interpretable changes in appearance. Starting from the embeddings of a source image ( left ), moving along early ( top ), intermediate ( middle ), and late ( bottom ) layers induces variations in geometry and illumination, gloss and style, or color, respectively.
Figure 6: Per-layer corrected conditional mutual information scores between generative factors and W+W+ layers, illustrating how different attributes are encoded across the latent hierarchy after accounting for the influence of the remaining factors.
Figure 7: Projection of samples onto a one-dimensional space (horizontal axis) learned with a linear ridge regression model. For all samples at each true gloss level (vertical axis), we show their scattering in the projected space, with small vertical jitter for visibility, together with the resulting half violin plot. Each point is projected using a model that was trained without that sample (cross-validation).
Figure 8: Representative 2D t-SNE visualizations illustrating the internal separation of samples within each label. Top: Early layers colored by geometry labels. Bottom: Late layers colored by color labels. Right: Layer 8 colored by style layers.
Figure 9: Uniqueness of gloss information per layer, measured by ΔR2\Delta R^{2} variations when predicting gloss labels without layer ii . Please refer to the text for more details.
#Styles2345
Reg ↓11.1318.4021.6822.13
PPL ↓54.3254.0360.2255.20
Table 1: Quantitative results for the ablation study on the number of styles. Reported metrics represent compactness ( Reg , lower is better), and smoothness ( PPL , lower is better) of the latent spaces.
Figure 10: Diagram of the proposed diffusion-based pipeline. It can handle up to three inputs: (i) a text prompt defining the color, geometry, and illumination, (ii) a style and gloss reference image that uses the intermediate layers of the latent space created in previous sections, and (iii) an optional image to guide spatial information of geometry and albedo.
Figure 11: Examples of three input modalities supported by our pipeline . Our method allows different levels of conditioning on geometry and color: color can be specified through a text prompt ( left ) or through spatial guidance using an albedo map ( middle ). Geometry is provided as edge or border maps, and by leveraging a ControlNet pretrained on Canny edges the pipeline also generalizes to other geometric representations such as clip-art drawings ( right ).
Figure 12: Qualitative comparison with state-of-the-art models . General-purpose T2I models produce visually striking images but often deviate from the target style and geometry. GPT Image 1.5 can often approximate the reference style, but its outputs offer only limited fine-grained control and may vary across prompts, making precise manipulation more challenging than with our method ( Fig. 13 ). Style-transfer methods better respect the reference style but tend to generate less compelling or less stable outputs. Our method achieves visually appealing results while maintaining high fidelity to both the reference geometry and the intended painterly style.
Figure 13: Comparison of gloss control . We compare our method against a diverse set of baselines: ConceptSliders [ 19 ] , a LoRA-based method; StyleID [ 34 ] , a training-free diffusion-based style-transfer method; GPT Image 1.5 [ 87 ] , a general-purpose image-generation model; and Artist-Inator [ 38 ] , the closest prior work to ours, which is based on ControlNet. For our method, the target gloss level is controlled with a slider, shown at the top of the figure. For Artist-Inator and GPT Image 1.5, it is specified through a text prompt, shown at the bottom and ablated in Sec. S8 of the supplementary material. For StyleID and ConceptSliders, gloss is varied by traversing a hyperparameter (see Sec. S7.2 for details). Our approach produces a clear, gradual modulation of gloss while preserving geometry and overall style. In contrast, the baselines have limited ability to produce smooth, continuous gloss changes while maintaining identity and fine details. GPT Image 1.5, in particular, often produces visually striking results but struggles to maintain a consistent gloss progression.
MethodMean Rank ↓RP ↓Pref. ↑
Ours1.88±1.46\mathbf{1.88\pm 1.46}1.534–
InstantStyle3.23±1.393.23\pm 1.392.91980.95%
StyleID4.02±2.254.02\pm 2.253.27476.59%
GPT Image 1.54.23±1.864.23\pm 1.863.71385.71%
Artist-Inator4.33±1.514.33\pm 1.514.03687.70%
FLUX4.69±1.704.69\pm 1.704.27387.30%
Table 2: Quantitative results for Experiment 1 (Style transfer), comparing our method with alternative style transfer approaches. We report the mean rank with standard deviation, the Rank Product (RP), and the preference rate of our method over each alternative (Pref.). Lower mean rank and RP indicate better performance, while higher preference indicates better performance of our model.
Figure 14: Distribution of preference rankings assigned to each evaluated method in Experiment 1, aggregated across all participants and the 12 evaluation examples. Each stacked bar shows the total number of times a method received each ranking position (R1–R7). Pairwise statistical comparisons reveal significant differences between methods, except for those enclosed in gray boxes, which are not statistically distinguishable.
MethodMAE ↓RMSE ↓%Exact ↑
Ours0.26±0.38\mathbf{0.26\pm 0.38}0.370.79
GPT Image 1.50.42±0.460.42\pm 0.460.580.66
ConceptSliders0.90±0.630.90\pm 0.631.180.48
Artist-Inator0.92±0.590.92\pm 0.591.210.43
Table 3: Quantitative results for Experiment 2 (Gloss ordering) across all trials. We report the mean absolute error with standard deviation, RMSE, and exact-match rate. Lower error and RMSE indicate better performance, while higher exact rate indicates better performance.
Figure 15: Bias in general-purpose models. Example illustrating how learned priors in large-scale text-to-image models can conflict with user prompts. In this case, GPT Image 1.5 struggles to generate glossy pineapples and instead introduces pineapple-like surface textures as gloss increases. In contrast, our disentangled representation enables more precise and continuous control over gloss while preserving the underlying object appearance.
More in this topic

Controllable Image Generation

Controllable image generation steers images using spatial, visual, semantic, stylistic, or physical constraints beyond ordinary text prompts.

See all 215 papers

Explore similar work

CardsList
Aug 5, 2026

StyleComposer: Training-Free Multi-Reference Style Composition

The style of a painting is not monolithic: color, texture, and structure may come from different sources. Existing reference-guided methods transfer them as one style signal, leaving each attribute's source and strength outside the user's control. We ask where in a diffusion model one attribute can change while the others hold, and find that no single representation isolates all three. The proposed StyleComposer therefore routes each style attribute through the representation where it separates best and coordinates the routes over denoising time. Without training or inversion, it satisfies three references and the prompt jointly more closely than prior methods, and exposes one strength slider per attribute. Project page: https://lexxsh.github.io/StyleComposer

Sanghyeok Lee, Jihye Kang, Namhyuk Ahn

Inha University

Image Style TransferCompositional T2I GenerationT2I Diffusion ModelsDiffusion-Based Denoising
AskPDF
Aug 12, 2026

Through Van Gogh's Eyes: Global Style Transfer with Diffusion Model

Artistic image synthesis aims to recreate the expressive visual identity of a target artist, yet existing methods often fail to capture an artist's global style. Conventional style transfer methods transfer the style of one or a few reference artworks to a content image in a One-to-One manner, making them effective for artwork-level stylization but limited in representing the broader stylistic distribution of an artist. Text-to-image diffusion models conditioned on artist names, such as '~ in Van Gogh style', offer greater flexibility, but they often suffer from text-induced bias and reproduce patterns from only a few iconic works. To address these limitations, we introduce Global Style Transfer (GST), an artistic image synthesis paradigm, in a Many-to-One manner, that aggregates multiple artworks from a target artist and transfers their shared global style to a single content image. For GST, we propose Global Style Guidance (GSG), which learns a residual global style offset in the intermediate feature space, or h-space, of a diffusion model under a fixed prompt. By learning artist-level style semantics purely from visual statistics, GSG mitigates text-dependent artistic bias. We further propose Content Alignment Guidance (CAG), a training-free perceptual guidance mechanism that preserves the semantic structure of the content image while allowing artist-specific geometric deformation. Experiments on WikiArt demonstrate that GST achieves superior stylistic fidelity, content preservation, and output diversity compared to existing style transfer and diffusion-based artistic synthesis methods.

Jeongha Lee, Yujin Kim, Ghazanfar Ali +2

Korea Institute of Science and Technology · University of Science and Technology · Korea University +2

Diffusion Model GuidanceImage Style TransferT2I Diffusion Models
AskPDF
Jun 1, 2026

Hist2Style: Histogram-Guided Stylization with Bilateral Grids

Photorealistic style transfer aims to match the color and tone of an input image to that of a style target while preserving the content and details of the original scene. Although existing large image models can facilitate these kinds of appearance edits, their high computational demands, potential for hallucinations, and limited user control make them unsuitable for high-resolution, real-time workflows. We introduce Hist2Style, a bilateral-grid formulation for fast, edge-aware stylization that preserves visual fidelity by constraining operations to locally affine transforms in bilateral space. Our model distills a large image editing model into a lightweight network by training on a large supervised corpus generated with language and vision-language models, targeting spatially varying color edits. The network conditions on a histogram-based embedding of the style target to provide an interpretable interface for adjusting the output style by modifying the target color distribution. Overall, Hist2Style maintains content structure by construction, avoids hallucinations, and supports real-time, high-resolution photorealistic stylization with interactive user-controllable color and tone adjustments.

Dekel Galor, Adam Pikielny, Zhoutong Zhang +4

Adobe Nextcam · University of California, Berkeley

Image Style Transfer
AskPDF