Resolution Allocation with Learnable Resizing and Spectral Filter Parameterization

Wei Wu; Eric C. Larson

cs.LGOct 14, 2026

Resolution Allocation with Learnable Resizing and Spectral Filter Parameterization

Authors: Wei Wu, Eric C. Larson

Organizations: Department of Computer Science, Southern Methodist University, Dallas, TX, USA

Abstract

Learnable downsampling and spectral-domain filter parameterizations have each made convolutional networks more flexible, but their interaction has not been systematically examined. We bridge this gap by examining convolutional kernels learned in the spatial and spectral domains, together with learned downsampling ratios under a cropped resizing implementation. We find that spectral and spatial kernels respond differently to learned resolution allocation, with larger gains for the spectral kernel. We observe this consistently on three datasets (EuroSAT, CIFAR-10 and STL-10) and at two model sizes (differences in accuracy gain of +0.013 to +0.028 on held-out test sets). Part of this difference reflects the greater gain of the spectral kernel from retaining more spatial area, but the difference remains positive when total retained area is matched. This pattern is observed across datasets and model sizes, although its magnitude and dependence on the complexity budget vary with model size and the resizing implementation.

Figures & tables

Figure 1: Gain from making

S

learnable, by filter type, under the cropped implementation (learnable resizing with feature maps cropped where the mask falls below 0.01). Held-out test accuracy relative to the fixed schedule at

p^{\star}=0.12

; points are individual seeds, horizontal bars are means and vertical bars are 95% confidence intervals (

n=10

). The spectral filters gain more in all four settings.

	Gain ( crop )		$\Delta_{\mathrm{gain}}$
	spatial	spectral	crop	ramp	soft
EuroSAT-S	$+0.009$	$+0.022$	$+0.013^{*}$	$+0.006$	$+0.020^{*}$
EuroSAT	$+0.004$	$+0.021$	$+0.016^{*}$	$+0.012^{*}$	$+0.022^{*}$
CIFAR-10	$-0.007$	$+0.011$	$+0.018^{*}$	$+0.002$	$+0.038^{*}$
STL-10	$+0.003$	$+0.031$	$+0.028^{*}$	$+0.008$	$+0.049^{*}$

Table 1: Gain from making

S

learnable at

p^{\star}=0.12

, held-out test accuracy (

n=10

). The first two columns are the gains under crop; the last three are

\Delta_{\mathrm{gain}}

under each implementation. EuroSAT-S uses the smaller models (

2.1\times 10^{4}

spatial and

2.5\times 10^{4}

spectral parameters); all other rows use approximately

5\times 10^{4}

parameters. ∗ : uncorrected

p<0.05

; all four crop comparisons remain significant after Holm correction.

Control	$\Delta_{\mathrm{gain}}$ ( $p$ )
Equal area, EuroSAT-S	$+0.006$ ( $0.08$ )
Equal area, EuroSAT	$+0.013$ ( $0.004$ )
Equal area, CIFAR-10	$+0.015$ ( $0.03$ )
Equal area, STL-10	$+0.009$ ( $0.13$ )
Spatial baseline, $F=8$ ( $12.7$ k params)	$+0.020$ ( $0.002$ )
Spatial baseline, $F=19$ ( $25.3$ k params)	$+0.023$ ( $0.002$ )

Table 2: Controls at

p^{\star}=0.12

. Top: crop compared with a fixed schedule of matched retained area (

p^{\star}=0.25

), validation. Middle: additional controls, test unless marked “val.”. Bottom: a fixed high-ratio schedule (

s_{1}/s_{2}=2.06

) against the default fixed schedule, test.

Figure 2:

\Delta_{\mathrm{gain}}

on EuroSAT under soft . (a) Across seven budgets at two model sizes (

n=20

per point, validation). (b) Across model size at

p^{\star}=0.12

(

n=10

, validation). Error bars are 95% confidence intervals.

Figure 3: Learned allocation and kernel size on EuroSAT at

p^{\star}=0.12

. (a) Mean adjacent-stage resolution ratios under soft over 10 seeds (separate runs), with the spectral kernel at stage 1 or 2; error bars are one standard deviation. (b) Effective area of the SpecMLP filters (640 per configuration) on the layer’s own grid at two energy thresholds; boxes show quartiles, whiskers the 5th–95th percentiles.

Convolutional Neural Networks

Convolutional neural networks (CNNs) learn spatial and temporal patterns using shared filters, with applications from image recognition to scientific prediction.

See all 411 papers

Useful to Whom? The Easy-versus-Coverage Boundary Moves with the Learner
Sep 22, 2026Yangze Liu, Xiao-Long Yin, Zhongyi HanTraining Data Selection Coreset Selection
Shandong University · Nanjing University
UCAN: Unified Convolutional Attention Network for Expansive Receptive Fields in Lightweight Super-Resolution
Mar 12, 2026Cao Thien Tan, Phan Thi Thu Trang, Do Nghiem Duc +3Image Super-Resolution Lightweight CNNs
Ho Chi Minh City Open University · AI Tech Lab, Ho Chi Minh City University of Technology · Code Mely AI Research Team +3
When does training on downscaled images yield the same gradients?
Aug 5, 2026Seunghyun JiDiffusion Transformer Diffusion Model Fine-Tuning
Independent researcher

Sep 22, 2026J/K move · Enter open · S save

Useful to Whom? The Easy-versus-Coverage Boundary Moves with the Learner

Yangze Liu, Xiao-Long Yin, Zhongyi Han

Shandong University · Nanjing University

Training Data Selection Coreset Selection Data Valuation

Open paper AskPDF arXiv

Is the same data equally useful to every learner? The question is whether the value of a training sample belongs to the sample or arises in its interaction with the learner. We test it with a selection experiment. Under a per-class budget, two kinds of samples compete for the slots: easy samples, which a short training run learns first, and coverage samples, which spread over the class in feature space. Easy samples win at small budgets and coverage samples at large ones, and the budget at which the winner flips, the crossover m0, can be measured. We ask whether m0 is set by the data or by the learner. To separate the two, we freeze the selected subsets and change only the learner that trains on them. On low-resolution ImageNet-100, doubling the width of a ResNet-18 moves the crossover from 57 to 85 samples per class on the same subsets. Enlarging the input grid from 32 to 64 pixels at fixed image information moves it from 85 to 57, and a single stride change reproduces or removes this shift. The grid shift shrinks as width grows. Under the native ImageNet-1k protocol, a 4x range of width moves the boundary by at most a few samples per class. On a Vision Transformer, coverage wins at every measured budget, even with easy subsets that work on the convolutional learner. The budget at which one kind of sample stops being the more useful kind therefore belongs to the learner, and the preferred regime of a selection strategy has to be read against the learner that will use the data.

Abstract

Convolutional Neural Networks

Explore similar work

Useful to Whom? The Easy-versus-Coverage Boundary Moves with the Learner

UCAN: Unified Convolutional Attention Network for Expansive Receptive Fields in Lightweight Super-Resolution

When does training on downscaled images yield the same gradients?