Deep Analog: Open-Set Film Emulation with Reference-Conditioned 3D LUTs

Yitong Mu

cs.CVAug 10, 2026

Deep Analog: Open-Set Film Emulation with Reference-Conditioned 3D LUTs

Authors: Yitong Mu

Organizations: Rochester Institute of Technology Rochester, NY 14623

Abstract

Film emulation reproduces the look of an analog film stock on a new digital photograph. We target its open-set form -- matching any reference film frame from a single example -- with a 3D lookup table (LUT) predicted from that reference. Real-time image enhancement predicts per-image weights over a fixed bank of 3D LUTs and blends them. We show this is a gated mixture of experts and inherits its failure: trained end-to-end against reconstruction, the gate collapses onto a single expert, so a bank of K LUTs delivers the capacity of one. An entropy term, the enhancement-setting analogue of mixture-of-experts load balancing, restores utilization and recovers about 1 dB PSNR. The deeper constraint survives: a fixed LUT basis is closed-set, freezing the achievable looks at training time. We therefore discard the basis and predict a single 3D LUT as a residual from a reference image (StyleLUTNet), trained by self-supervision on procedurally generated color transforms. The conditional design removes the gate and generalizes open-set to unseen film stocks without paired data or retraining. Around this color backbone we build Deep Analog, a film-emulation pipeline that adds histogram-based tone matching and a physics-informed optical renderer -- multi-scale grain and per-channel halation driven by parameters an inverse network regresses from the reference. On 350 self-supervised pairs the color stage reaches 22.05 dB PSNR / 0.925 SSIM and the full pipeline 21.72 dB / 0.923; the color path runs in 5.2 ms at 1080p (192 FPS) and exports a portable .cube LUT for standard editing tools. A second degeneracy in conditional LUT training -- residual-scale collapse -- shares the root cause and yields a general principle: auxiliary regularization must stay subordinate to reconstruction.

Figures & tables

Figure 2: Deep Analog architecture. StyleLUTNet encodes a resized reference scan into a residual $17^{3}$ LUT, adds it to a fixed identity LUT, and upsamples the result to $33^{3}$ . In parallel, FilmAnalyzerNet predicts seven groups of physical parameters (11 scalar outputs), while a deterministic analyzer measures reference CDFs, black and white points, and shadow/highlight color casts. The predicted LUT, reference statistics, and physical parameters condition the color, tone, and optical components of the renderer, respectively. The system exports the LUT as a .cube file and combines tone statistics with physical rendering parameters in a .json preset.

Figure 3: Stagewise processing workflow for Rainy Night NYC conditioned on an Escura Showa film scan. The reference card includes the predicted 3D LUT, whose point positions and colors encode output RGB coordinates. The numbered sequence follows a two-row path: input, StyleLUT grading, CDF tone-curve matching, analytic film-tone processing, grain synthesis, and halation rendering. Each card pairs the image with its RGB histogram; diagnostic panels beneath the corresponding transition arrows visualize the CDF-derived tone-transfer curve, grain residual, and halation map. Grain and halation use illustrative amplified parameters for print visibility.

Figure 4: Analytic tone response (Section 3.4.2 ). The black-point lift (blue) raises the toe; the quadratic highlight rolloff (orange) softens the shoulder; together (black) they compress the tonal range into the lifted-black, rolled-highlight response characteristic of scanned film, using two parameters ( $b_{\text{ref}}$ , $w_{\text{ref}}$ ) read from the reference. This step is applied after the neural LUT and CDF matching, which carry the bulk of the tonal transfer, so it remains a small residual correction—unlike the display-referred operators (Reinhard [ 22 ] , Naka–Rushton [ 23 ] , ACES filmic) that a full sigmoid fit would require.

Figure 5: Response functions in the physics-informed renderer. (a) Per-channel CDF transfer curves computed for the Rainy Night NYC example after StyleLUT grading, conditioned on the Escura Showa reference; the dashed diagonal is the identity mapping. (b) The luminance-dependent grain mask

M(L)=\sigma(aL^{2}+bL+c)

for

a=-2.0

b=1.5

, and

c=0.8

, showing strongest response in the mid-tones. (c) Normalized radial profiles of the per-channel halation kernels. The red kernel has the broadest support (

1.4r

), followed by green (

r

) and blue (

0.7r

), producing the characteristic warm halo around bright regions.

Figure 6: Controlled demonstration of the optical renderer on synthetic test scenes, applying the grain (Section 3.4.3 ) and halation (Section 3.4.4 ) equations directly. These illustrate the rendering functions in isolation; they are not results on real film. Top row (grain): (a) a clean luminance ramp; (b) after multi-scale grain, whose amplitude is modulated by the luminance mask $M(L)$ ; (c) the grain layer amplified $6\times$ . Bottom row (halation): (d) bright discs on a dark background; (e) after halation, the highlights acquire a warm glow that spreads beyond their borders; (f) the halation layer amplified, showing the red-biased spread produced by the per-channel radii $r_{R}\!:\!r_{G}\!:\!r_{B}=1.4\!:\!1.0\!:\!0.7$ .

Figure 7: Anatomy of multi-LUT weight collapse. (a–b) Gate distributions over all 350 FiveK test images. V1 assigns a mean weight of 0.947 to LUT 1 (entropy 0.16), whereas entropy-regularized V3 uses all five bases (entropy 1.58; maximum 1.61). (c–d) Checkpoint-wise mean absolute LUT coefficients at 10-epoch intervals. In V1, only LUT 1 develops substantial magnitude while four bases remain effectively inactive; in V3, every basis evolves during training. (e) Central $B=0.5$ slices of the final basis LUTs, with mean test-set gate weights $\bar{w}$ . The suppressed V1 bases remain nearly black, while V3 learns five distinct color mappings.

Metric	V1	V2	V3
Test PSNR (dB)	21.27	22.19	22.16
Test SSIM	0.903	0.929	0.927
Weight Entropy	0.158	1.588	1.577
Overfit Gap (dB)	0.57	3.77	2.27
Dominant LUT Wt.	0.947	0.208	0.263
Data Augmentation	—	—	✓

Table 1: Progression across SwinLUT model versions on MIT-Adobe FiveK (Expert C, 350 test images).

Configuration	PSNR	SSIM	Entropy
Full V3	22.16	0.927	1.577
No Perceptual	21.86	0.922	1.569
No Entropy Reg.	20.96	0.905	0.000
No Diversity	21.70	0.917	1.571
L1 Only	20.89	0.905	0.000

Table 2: Ablation study: effect of disabling loss components in SwinLUT. Ablation variants trained 30 epochs with the V3 architecture; best validation metrics reported. Full V3 is the fully trained model of Table 1 , shown for reference.

Configuration	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Naive (no processing)	21.19	0.915	0.517
StyleLUT only	22.05	0.925	0.308
+ Grain	21.82	0.923	0.415
+ Grain + Halation	21.76	0.923	0.416
V6 (E2E, all stages)	21.72	0.923	0.416

Table 3: End-to-end pipeline ablation. Each row adds one processing stage to the previous. Evaluated on 350 self-supervised test pairs. V6 jointly fine-tunes the stock-conditioned SwinLUT, GrainNet, and HalationNet predecessor modules; earlier rows use independently trained modules.

Figure 8: Per-stage inference latency on an NVIDIA RTX 4060 GPU (median of 20 runs). The logarithmic axis separates the resolution-independent LUT encoder from pixel-count-dependent stages. The dashed line marks the 16.7 ms budget for 60 FPS. Tone-curve matching is the dominant bottleneck, while StyleLUT encoding and application total 5.2 ms at 1080p and 12.0 ms at 4K.

	1080p		4K
Device	StyleLUT	Full	StyleLUT	Full
RTX 4060 (CUDA)	5.2	198.9	12.0	1052.9
Apple MPS	11.1	510.1	27.7	1803.4
CPU (Apple M-series)	102.0	1783.0	359.4	6323.7

Table 4: Cross-device comparison at 1080p and 4K (ms). RTX 4060 provides the fastest absolute times; Apple Silicon MPS on a MacBook Pro (2023) offers a portable alternative.

Figure 9: Open-set generalization across diverse film stocks and scene types. Each row shows a reference film scan, its reference-conditioned 3D LUT, a digital input photograph, and the Deep Analog output after applying the inferred look. Point positions and colors in each LUT visualization encode output RGB coordinates. References, top to bottom: Once Upon a Time in the West (1968), Happy Together (1997), Days of Being Wild (1990), Barry Lyndon (1975). Digital inputs are original photographs by the author. The system reproduces each reference’s characteristic color palette and tonal character without film-stock-specific training.

Figure 10: Multi-reference film look transfer across five film stocks and seven input scenes. The top two rows show each uncropped reference film scan and its corresponding reference-conditioned 3D LUT; point positions and colors encode LUT output RGB coordinates. References were captured by the author on Kodak Vision3 5207 250D, Escura Showa Camera, Escura Edo Ukiyo-e 400, Kodak Gold 400, and Fujifilm 400. The separated leftmost column contains uncropped digital inputs spanning daylight, low-light, indoor, portrait, and nature conditions. Each result cell preserves the source aspect ratio and shows the Deep Analog output conditioned on the corresponding reference—all produced by a single model without film-stock-specific fine-tuning. All photographs are by the author.

Figure 11: Close-up comparison of grain synthesis and halation rendering. Top row (grain): (a) Full output with crop region marked in green, (b) a white bottle surface before grain, (c) the same region after grain synthesis—note the fine noise texture on the previously smooth surface, (d) grain layer amplified $8\times$ , showing luminance-dependent density variation. Bottom row (halation): (e) Full output (rainy night scene) with crop region marked in orange, (f) wet pavement reflections before halation, (g) after halation—the bright reflections develop a warm glow that spreads across the road surface, (h) halation map amplified $5\times$ , revealing the per-channel color spread: the red channel, which dominates the base-reflected re-exposure, spreads with a larger radius than blue. Halation parameters are amplified ${\sim}3\times$ relative to defaults for print visibility. Both images: digital photographs by the author; reference: Kodak Gold 400 film scan.

Method	Params	LUTs	Conditioning	PSNR	SSIM
3D LUT [ 1 ]	594K	3 $\times$ 33 3	None (fixed)	25.28	0.926
AdaInt [ 3 ]	620K	3 $\times$ 33 3	None (fixed)	25.50	0.930
SepLUT [ 4 ]	$\sim$ 280K	3 $\times$ 1D	None (fixed)	—	—
StarEnhancer [ 2 ]	—	3 $\times$ 33 3	Style embedding	—	—
SwinLUT V3 (ours)	28.3M	5 $\times$ 33 3	None (fixed)	22.16	0.927
StyleLUTNet (ours)	82.6M	1 $\times$ 33 3	Ref. image	22.05 †	0.925 †

Table 5: Comparison with LUT-based baselines. Closed-set methods learn a fixed enhancement mapping (FiveK Expert C, 480p). StyleLUTNet is open-set, evaluated on self-supervised pairs (

\dagger

). Baseline values are from the respective publications.

Evaluator	Part 1 mean	Part 2 acc.
E1	3.17	10/16
E2	3.03	0 9/16
E3	3.60	10/16
E4	3.23	0 9/16
Pooled	3.26 (SD 0.94)	38/64 = 59.4%

Table 6: Expert evaluation (

n\!=\!4

imaging professionals). Part 1: mean reference-match rating (1–5). Part 2: accuracy at telling real film from Deep Analog output; 50% is chance. With only four evaluators, counts are reported descriptively, without an inferential test.

Figure 12: Boundary cases selected from the 35 open-set combinations. Columns show the reference, input, output, and selection diagnostic. (a) Saturation amplification. (b) Highlight clipping, with luminance above 0.97 in red. (c) Tone compression, measured by the 5th–95th percentile luminance range. Statistics use complete, uncropped images.

Panel of Figure 13 — Figure 13: The public web application. (a, top) With a target photograph, the rendered/original comparison sits above the export row ( .cube LUT, Adobe .xmp , Capture One .costyle , JSON parameters); below, the look recipe shows the reference as the model saw it, its extracted palette, per-hue color ribbons, an interactive rendering of the predicted $33^{3}$ LUT, and the regressed grain parameters. (b, bottom) Reference-only mode previews the predicted LUT on a neutral color chart, with the same export row. Exposing these intermediates in the browser makes each pipeline stage inspectable.

Image-to-Image Translation

Image-to-image translation maps images between visual domains or modalities while preserving relevant content, structure, and semantics.

See all 115 papers

Feb 26, 2026

LoR-LUT: Learning Compact 3D Lookup Tables via Low-Rank Residuals

We present LoR-LUT, a unified low-rank formulation for compact and interpretable 3D lookup table (LUT) generation. Unlike conventional 3D-LUT-based techniques that rely on fusion of basis LUTs, which are usually dense tensors, our unified approach extends the current framework by jointly using residual corrections, which are in fact low-rank tensors, together with a set of basis LUTs. The approach described here improves the existing perceptual quality of an image, which is primarily due to the technique's novel use of residual corrections. At the same time, we achieve the same level of trilinear interpolation complexity, using a significantly smaller number of network, residual corrections, and LUT parameters. The experimental results obtained from LoR-LUT, which is trained on the MIT-Adobe FiveK dataset, reproduce expert-level retouching characteristics with high perceptual fidelity and a sub-megabyte model size. Furthermore, we introduce an interactive visualization tool, termed LoR-LUT Viewer, which transforms an input image into the LUT-adjusted output image, via a number of slidebars that control different parameters. The tool provides an effective way to enhance interpretability and user confidence in the visual results. Overall, our proposed formulation offers a compact, interpretable, and efficient direction for future LUT-based image enhancement and style transfer.

Ziqi Zhao, Abhijit Mishra, Shounak Roychowdhury

The University of Texas at Austin, Austin, TX, USA

Image Enhancement Image Editing

Ask PDF

Sep 27, 2026

LoopLUT: 3D Lookup Tables with Progressive Region Refinement for Real-Time 4K Image Enhancement

Color enhancement of 4K images must meet a quality target under a tight compute budget. Three-dimensional lookup tables (3D LUTs) dominate real-time enhancement because they decide at low resolution and apply a per-pixel lookup at full resolution. A single global LUT, however, is spatially invariant, so an underexposed shadow and a well-exposed region that share a pixel value receive identical corrections. Spatially heterogeneous demands cannot be expressed by such a mapping. We propose LoopLUT, a region-cascaded 3D LUT with progressive refinement. A global LUT performs the overall correction, followed by K-1 loop iterations. In each iteration a gating head predicts at low resolution the region that still needs correction, then builds a residual LUT from the color statistics of that region alone. The cascaded gates form a partition of unity, so the output is a per-pixel convex combination of the K lookup results. Fusion is therefore performed by the gates themselves, with no separate fusion module and no interpolation error accumulating across rounds. The decision stage runs at a fixed 256x256 resolution, independent of output resolution, so a 4K image costs only K pure lookups. Extensive experiments across four benchmarks show that LoopLUT improves PSNR by up to 2.81 dB over the strongest prior method, while keeping real-time throughput at 4K. The same decomposition also generalizes well to underwater enhancement datasets.

Yang Ye, Jiajun Ma, Chen Wu +5

Universiti Sains Malaysia · National University of Defense Technology · Sun Yat-sen University +2

Image Enhancement

Ask PDF

Sep 28, 2026

PrefLUT: Reusable and Refinable Personalized Color Editing from Pairwise Preferences

Photographic color editing is inherently personal: the same image can appear too warm, too muted, or already satisfactory to different users. Most lookup table (LUT) and reference-guided methods target a specified appearance rather than model persistent preferences from repeated user choices. To address this gap, we introduce PrefLUT, a reusable and refinable user-preference modeling framework for deployable 3D LUTs, encoding ordered preferred/non-preferred image pairs into a lightweight Reusable User Profile that is reused across queries and refined using additional user preference pairs, without per-user optimization. A Query-Conditioned LUT Predictor combines this profile with each image to predict a LUT latent vector and edit strength. An Identity-Residual LUT Decoder and Edit-Strength Controller then produce an exportable 3D LUT. Experiments on three datasets demonstrate effective personalized editing and general-purpose enhancement. Each quantized profile requires only 260 bytes, and editing takes 1.365 ms/image on an RTX 5090 GPU. We also introduce the Preference-Conditioning Verification Protocol (PCVP), an evaluation protocol to verify whether personalized image edits depend on user preferences and the query image through controlled changes to user profiles, preference orders, pair correspondences, and query images.

Chuanzhi Xu, Langyi Chen, Chengkun Yue +5

The University of Sydney · Indiana University · The University of Hong Kong

Pairwise Preference Learning User Preference Modeling Image Editing

Ask PDF

Abstract

Image-to-Image Translation

Explore similar work

LoR-LUT: Learning Compact 3D Lookup Tables via Low-Rank Residuals

LoopLUT: 3D Lookup Tables with Progressive Region Refinement for Real-Time 4K Image Enhancement

PrefLUT: Reusable and Refinable Personalized Color Editing from Pairwise Preferences