A Unified Theory of Image Scaling: Point Spread Functions, Discrete Kernels, and Scale-Spaces

Assaf Shocher

eess.IVOct 15, 2026

A Unified Theory of Image Scaling: Point Spread Functions, Discrete Kernels, and Scale-Spaces

Authors: Assaf Shocher

Organizations: Technion – Israel Institute of Technology

Abstract

We derive the optically aligned system of image-scaling operators $\mathbf{K}_s$ that reproduces optical zoom (exact zoom-out, canonical zoom-in), by identifying a Huygens-like consistency principle in which each pixel acts as a secondary source. The system is determined in closed form by the camera's point spread function (PSF), itself the operator $\mathbf{K}_0$ . The operators compose, $\mathbf{K}_{s'} \circ \mathbf{K}_s = \mathbf{K}_{ss'}$ (with band projection for down-up), and account for both upscaling and downscaling, which form a Moore-Penrose pseudoinverse pair, $\mathbf{K}_{1/s} = \mathbf{K}_s^\dagger$ . We show that image scaling, as routinely practiced, is neither optically faithful nor internally consistent; for example, common anti-aliasing rules generally correspond to no camera, and two resizes by a factor of $1/2$ generally do not equal one resize by a factor of $1/4$ . Gaussian scale-space kernels cannot be exact finite-aperture PSFs. Gaussian scale-space is an idealized special case of what emerges here: a generalized scale-space for sampled images, derived entirely from camera optics, bidirectional, and not limited to Gaussian kernels. The system goes beyond scaling and supports composable and invertible subpixel geometric transformations. We empirically validate its optical predictions with real captures and noisy optics simulations, and demonstrate its practical impact on the stability of tasks such as super-resolution and classification.

Figures & tables

Figure 1: A Huygens-like picture of bidirectional scaling.

\!s\!<\!1\!

. Pixels act as secondary sources whose signals superpose to reconstruct a band-limited scene. Rescaling and recapturing it with the same camera (top) reproduces optical zoom-out (middle). The reverse route (bottom, right to left) restores the original resolution, not discarded frequencies. Details and derivation at Section 3 .

Figure 2: The four phases of resizing: output pixels project to sub-pixel locations (crosses), gather neighbors, weight them by chosen interpolation method evaluated at signed offsets (blue positive, red negative) and sum.

Figure 3: The four phases of Fig. 2 as one continuous process [ 35 ] : lifted to a bed of nails, convolve with a continuous kernel, resample.

Figure 4: Three systems in four views of $K_{s}(x)$ over position and scale (signed-log height). Red: assumed slice (first two) or PSF (third). At $s=1$ , $K_{s}$ is a band-limited delta, not a Dirac (Proposition 4.7 ). In output pixels (third), coarsening approaches the PSF; in input pixels (rightmost), the same Gaussian camera ( $\sigma=1$ ) shows observed-band Gaussian coarsening and band-limited deconvolution for $s>1$ (Remark 5.2 ). Galleries: App. X .

	kernels	outputs	convolution?
down, full visible band	one	one	yes
down, partial visible band	many, consistent	one	yes
up, within the band	many, consistent (one iff full band)	one , band-limited interpolation	yes
up, adding frequencies	many, none consistent	many , $\mathbf{K}_{s}I+\operatorname{null}\mathbf{K}_{\nicefrac{{1}}{{s}}}$	not necessarily

Table 1: Uniqueness: closed working band, dilation-nested visibility. Null spaces permit many kernels but one output; new frequencies allow many outputs. “Full band”:

\widehat{K}_{0}\neq 0

a.e. on

[-\pi,\pi]

Figure 5: Many kernels, one output (

B=0.6\pi

). Top: canonical (black), consistent alternatives (red), and replica-reading nonmembers (orange) at

\times\frac{1}{3}

and

\times 3

. Members share taps at

\times\frac{1}{3}

, but not at

\times 3

. Bottom: consistent kernels give one output; nonmembers add fine-Nyquist aliases.

Figure 6: Cycle, composition, and commutativity tests. (a) Up $\times 2$ then down $\times 2$ should return the input: bicubic kernel model (top), defocus-disc camera (bottom). The baseline interpolates with the declared kernel and downsamples with its AA stretch; the disc baseline is constructed, not a library method. Ours applies $\mathbf{K}_{2}$ then $\mathbf{K}_{1/2}$ . (b) Two unequal, non-integer reductions versus one. (c) The same reductions in either order. Both (b,c) use one shared $\times\frac{1}{2}$ bicubic kernel, band-limited at $0.99\pi$ . The baselines fail; ours close to numerical precision. More cameras and longer chains: Figs. 19 , 19 .

Figure 7: Geometric transformation test. A capture is rotated

15^{\circ}

and shifted

(0.37,0.21)

px, then inverted. Model and details: App. M .

Figure 8: Optical test. Left: capture spectrum ratio at

d,2d

. Right: prediction/composition errors (finite grid); dashed: noise reference. Matched AA dilates

K_{1/2}

; Fourier is flat on the same band.

Figure 9: Scale-law robustness in repeated learned SR: BSD100 mean Y-PSNR vs step count. AA (PyTorch grid) changes only downsampling grid/size; upsampling remains ResizeRight. Gray: one-shot bicubic. Vertical ranges differ. Set5/Set14, protocol, and controls: App. W .

		per-pair fit	consistent	AA stretch
Canon 5D3	$\times 3$	32.9	31.5	31.3
	$\times 4$	31.8	29.9	29.5
Nikon D810	$\times 3$	33.0	31.6	31.0
	$\times 4$	32.2	30.7	30.2

Table 2: RealSR: median PSNR over 50 scenes per camera. Kernels transfer from

\times 2

with target-pair phase/gain calibration. Per-pair fits are references, not held-out ceilings (App. P ).

Method	G $\times 2$	G $\times 3$	G $\times 4$	B $\times 2$	B $\times 3$	B $\times 4$
Cubic	25.80	25.59	25.27	29.67	27.22	25.98
Lanczos	25.88	25.74	25.44	29.99	27.42	26.14
Fourier	25.87	25.78	25.50	30.07	27.45	26.17
Canonical	30.65	27.62	26.21	30.10	27.47	26.20

Table 3: One-shot upscaling: mean BSD100 Y-PSNR (dB), float inputs. G: Gaussian (

\sigma=2

HR pixels); B: bicubic AA. Lanczos-3; ideal Fourier. Only the canonical operator knows the blur. Set5/Set14 and protocol: App. S .

Figure 10: Resize-path dependence on 50 crops, $2047\to 255$ directly or in two to five steps. Left: signed, peak-normalized $K_{1/2}$ , not PSFs; the three camera families are declared controls, not scene-camera estimates. (a) NRMSE to each method’s direct output. (b) Five-step classifier prediction changes, with $95\%$ scene-bootstrap intervals. Protocol: App. N ; longer chains: Fig. 19 .

Figure 11: HPatches down–up test. Reduce and restore images before SIFT homography estimation; success: median error below five original pixels against supplied truth. Right: consistent-minus-AA gain with

95\%

sequence-cluster bootstrap intervals. The consistent pair realizes ideal retained-band projection for every base. Protocol: App. O .

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 12: The projected grid. (a) Scale fixes spacing; centering fixes

q

. Crosses show the output centers on the input row. (b) When

N\neq sN_{0}

, the half-pixel and first-center shortcuts move the optical axis. (c) A size-changing round trip needs both its physical scale and target size.

Figure 13: Noise-aware canonical refinement. Mean unclipped Y-PSNR over five Set5 crops and three noise realizations against the input-determined optical target. Tikhonov regularization suppresses noise amplification near weak camera frequencies. No learned model or missing-frequency prediction is used. The noiseless numerical identity is reported in the text, not capped on the plot.

Degradation	Scale	Bilinear	Bicubic	Lanczos-3	Fourier interp.	Camera-derived
Gaussian	$\times 2$	27.36	27.78	27.90	27.88	32.60
	$\times 3$	26.93	27.50	27.77	27.80	31.24
	$\times 4$	26.28	27.04	27.39	27.49	29.00
Bicubic AA	$\times 2$	32.26	33.69	34.33	34.54	34.77
	$\times 3$	29.50	30.42	30.86	30.96	31.06
	$\times 4$	27.56	28.43	28.82	28.92	29.01

Appendix

Table 4: Full clean-upscaling comparison, mean Set5 Y-PSNR in dB. Only camera-derived refinement uses the known degradation. All scores use the same observation and border.

Degradation	Scale	Bilinear	Bicubic	Lanczos-3	Fourier interp.	Camera-derived
Gaussian	$\times 2$	25.46	25.76	25.86	25.85	29.81
	$\times 3$	25.11	25.53	25.72	25.77	28.26
	$\times 4$	24.64	25.16	25.39	25.43	26.34
Bicubic AA	$\times 2$	29.34	30.37	30.82	30.93	31.06
	$\times 3$	27.00	27.65	27.95	28.03	28.12
	$\times 4$	25.54	26.11	26.33	26.34	26.36

Appendix

Table 5: Full clean-upscaling comparison, mean Set14 Y-PSNR in dB. Only camera-derived refinement uses the known degradation. Same floating-point protocol as Table 4 .

Figure 14: Aliasing stress test, clean textured ensemble, five scenes. Left: prediction of an independently rendered quarter-scale capture. Right: two half-steps versus one quarter-step. The projected camera law remains algebraically consistent but ceases to reproduce aliased optical captures. The full-band ratio predicts these captures more accurately here, but is only a heuristic and loses composition. The vertical line marks sensor Nyquist; values below

10^{-13}

are displayed at that numerical floor.

Figure 15: Set5 repeated learned SR, corresponding to Fig. 9 . Mean Y-PSNR over five images, one seed; gray is one-shot bicubic. Vertical ranges differ. Protocol: App. W .

Figure 16: Set14 repeated learned SR, corresponding to Fig. 9 . Mean Y-PSNR over 14 images, one seed; gray is one-shot bicubic. Vertical ranges differ. Protocol: App. W .

Figure 17: Learned SR from ordinary bicubic-AA observations. Same Set5 protocol described above, without band-limiting the observation kernel. Ours uses the untapered camera ratio on the retained band, so its anchor is approximate: the one-step results need not coincide. AA wins at early steps; ours is more robust over longer chains, but does not attain the best mean PSNR over the sweep. Curves average all five images, one seed; gray is one-shot bicubic interpolation. AA (PyTorch grid) changes only downsampling grids, not the shared ResizeRight upsampler. Vertical ranges differ.

Panel of Figure 18 — Figure 18: The chain test: $\times\frac{1}{8}$ in forty small steps against one step, on a photograph and a zone plate. Right: detail kept at the output Nyquist against the number of steps.

Computational Imaging

Computational imaging combines optical and sensor design with algorithms to recover or infer scene information from indirect, compressed, or noisy measurements.

See all 123 papers

Multiscale Super Resolution without Image Priors
Apr 23, 2026Daniel Fu, Gabby Litterio, Pedro Felzenszwalb +1Image Super-Resolution Image Inverse Problems
Division of Applied Mathematics (Fu) and School of Engineering (Litterio, Felzenszwalb, Zia), Brown University, Providence, RI, 02906.
CASISR: Circular Arbitrary-Scale Image Super-Resolution
May 5, 2026Honggui Li, Zhengyang Zhang, Dingtai Li +11Image Super-Resolution
School of Information and Artificial Intelligence, Yangzhou University, Yangzhou 225127, China · Shanghai Qigong Research Institute, Shanghai University of Traditional Chinese Medicine, Shanghai 201203, China · LISITE Research Laboratory, Institut Supérieur d'Électronique de Paris, Paris 75006, France +4
Is Zero-Shot Super-Resolution Possible in Operator Learning?
May 29, 2026Unique Subedi, Ambuj TewariOperator Learning Statistical Learning Theory
Department of Statistics, University of Michigan

Apr 23, 2026J/K move · Enter open · S save

Multiscale Super Resolution without Image Priors

Daniel Fu, Gabby Litterio, Pedro Felzenszwalb, Rashid Zia

Division of Applied Mathematics (Fu) and School of Engineering (Litterio, Felzenszwalb, Zia), Brown University, Providence, RI, 02906.

Image Super-Resolution Image Inverse Problems

Open paper AskPDF arXiv

We address the ambiguities in the super-resolution problem under translation. We demonstrate that combinations of low-resolution images at different scales can be used to make the super-resolution problem well posed. Such differences in scale can be achieved using sensors with different pixel sizes (as demonstrated here) or by varying the effective pixel size through changes in optical magnification (e.g., using a zoom lens). We show that images acquired with pairwise coprime pixel sizes lead to a system with a stable inverse, and furthermore, that super-resolution images can be reconstructed efficiently using Fourier domain techniques or iterative least squares methods. Our mathematical analysis provides an expression for the expected error of the least squares reconstruction for large signals assuming i.i.d. noise that elucidates the noise-resolution tradeoff. These results are validated through both one- and two-dimensional experiments that leverage charge-coupled device (CCD) hardware binning to explore reconstructions over a large range of effective pixel sizes. Finally, two-dimensional reconstructions for a series of targets are used to demonstrate the advantages of multiscale super-resolution, and implications of these results for common imaging systems are discussed.

Abstract

Computational Imaging

Explore similar work

Multiscale Super Resolution without Image Priors

CASISR: Circular Arbitrary-Scale Image Super-Resolution

Is Zero-Shot Super-Resolution Possible in Operator Learning?