Beyond the Grid: Continuous Dictionary Pursuit for Interpretable Signal Decomposition

Nick Janssen; Melanie Schaller; Bodo Rosenhahn

cs.LGNov 6, 2026

Beyond the Grid: Continuous Dictionary Pursuit for Interpretable Signal Decomposition

Authors: Nick Janssen, Melanie Schaller, Bodo Rosenhahn

Organizations: Institute of Information Processing, Leibniz University Hannover

Abstract

Deep learning models achieve high accuracy in signal processing but remain difficult to interpret. Classical matching pursuit and sparse coding offer better transparency but suffer from spectral leakage and grid mismatch when signal frequencies do not align with discrete dictionary atoms. We introduce Continuous Dictionary Pursuit (CDP), a framework for interpretable signal decomposition that learns the parameters of closed-form functions directly from data. CDP optimizes over a continuous manifold of differentiable atoms and, unlike prior continuous pursuit methods that operate on a single analytic family, searches a heterogeneous library spanning periodic, trend, transient, and discontinuous structures. The algorithm employs a greedy iterative strategy using data-driven priors and gradient-based optimization to isolate individual signal components, with discontinuous waveforms made differentiable through spectral sharpness annealing. To ensure a sparse atom set for reconstruction, the framework utilizes a stopping criterion based on a predefined atom budget and residual error convergence. Evaluations on synthetic and real-world datasets show that CDP removes the grid discretization error for atom families contained in its library, recovers generative parameters to within $1\%$ on stationary synthetic signals expressible in that library, and achieves the best sparsity-accuracy trade-off among classical, continuous-pursuit, and neural baselines in the sparse regime. Beyond decomposition, we demonstrate that the extracted atoms can serve as structured curriculum targets for training sequence models, improving forecasting accuracy over standard end-to-end training on most evaluated datasets. The method offers a glass-box alternative for signal decomposition by providing high reconstruction fidelity alongside direct physical interpretability.

Figures & tables

Figure 1: Continuous Dictionary Pursuit: The iterative greedy procedure initializes atoms from data-driven priors, identifies a candidate via truncated optimization, fully optimizes it with L-BFGS [ 30 ] , and adds it to the additive model until the atom budget is exhausted or the residual error stagnates.

Figure 2: Parametric Dictionary Library: Example parameterizations of atom types: (a) SineAtom, (b) LinearAtom, (c) ExponentialAtom, (d) GaussianAtom, (e) SquareAtom, (f) SawtoothAtom, (g) StepAtom, (h) ModulatedAtom (for example

\texttt{SineAtom}\cdot\texttt{StepAtom}

Figure 3: Parameter recovery on synthetic datasets: CDP reconstructs the true mathematical components from noisy targets (

\sigma=0.1

): (a) linear trend, (b) sine, (c) square wave, (d) off-grid frequency superposition, (e) damped sine via a modulated atom, (f) non-stationary amplitude shift. Reconstruction and prediction MSE are given per panel.

Figure 4: Pareto frontier of parameter count versus training reconstruction MSE: CDP and CDP (Modulated) achieve the optimal trade-off in the sparse regime across all baselines. The lower convex envelope excludes Analysis K-SVD, whose near-zero training residual reflects in-sample denoising rather than a parametric decomposition ( subsection A.8 ).

Dataset	NOMP	SFW	MCA	AK-SVD	CDP-sine	CDP
Electricity	0.562	0.567	0.622	0.007	0.593	0.514
ETTh1	0.418	0.421	0.594	0.005	0.486	0.427
ETTh2	0.539	0.545	0.760	0.004	0.692	0.545
ETTm1	0.420	0.422	0.575	0.002	0.488	0.426
ETTm2	0.537	0.542	0.740	0.002	0.691	0.491
Exchange	0.055	0.060	0.135	0.000	0.145	0.066

Table 1: Per-dataset training reconstruction loss at

K=5

for the continuous and structured baselines, the CDP-sine ablation, and full CDP. The best value per dataset is bold, excluding Analysis K-SVD, whose near-zero training residual reflects in-sample denoising rather than a parametric decomposition ( subsection A.8 ).

Figure 5: Signal decomposition of the PEMS04 traffic dataset: Top: the original signal overlaid with the full CDP reconstruction. Left: the individual extracted atoms, capturing daily cycles, weekly patterns, and localized rush-hour modulations. Right: their progressive summation toward the target signal.

Figure 6: Forecasting MSE for curriculum learning versus standard end-to-end training across all 13 real-world LTSF datasets (IQM over 10 seeds, whiskers show 95% bootstrap confidence intervals).

Dataset	CDP	CDP (rev.)	SSA	MSTL	OMP	Prophet	NOMP	SFW	Smooth.	Standard
Electricity	1.281	1.277	1.286	1.295	1.269	1.273	1.254	1.269	1.261	1.270
ETTh1	2.128	2.244	2.272	2.285	2.216	2.199	2.222	2.241	2.283	2.345
ETTh2	1.409	1.675	1.566	1.449	1.503	1.496	1.497	1.397	1.492	1.593
ETTm1	2.006	2.105	2.162	2.158	2.100	2.177	2.065	2.136	2.183	2.052
ETTm2	1.233	1.272	1.233	1.218	1.227	1.254	1.220	1.285	1.236	1.228
Exchange	2.488	2.340	2.265	2.260	2.062	2.335	2.323	$9.8{\times}10^{4}$	1.824	2.248

Table 2: Curriculum evaluation on all datasets. Out-of-sample forecast MSE of the S4D model for every curriculum target (IQM over 10 seeds). The best value per dataset is bold.

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Component	Details	Default
Input projection	Conv1d( $1\to d_{\text{model}}$ , kernel 1)	$d_{\text{model}}=128$
S4D blocks	$n_{\text{layers}}$ stacked S4DBlocks	$n_{\text{layers}}=4$
State dimension	Per-block S4D state size	$d_{\text{state}}=64$
Local conv	Depthwise Conv1d(kernel 3, padding 1)	per block
Output gate	GLU via Conv1d( $d_{\text{model}}\to 2d_{\text{model}}$ )	per block
Dropout	Applied after GLU	$p=0.1$

Appendix

Table 3: AtomS4DModel architecture. Default hyperparameters used in all curriculum learning experiments.

Hyperparameter	Description	Default
$T$	Total training iterations	1000
$K$	Total number of atoms	10
$\rho$	Fraction of iterations on real data	0.1
$\alpha$	Curriculum transition smoothness	0.2

Appendix

Table 4: Curriculum learning hyperparameters used in all experiments.

Figure 7: Pareto frontier of model complexity versus out-of-sample forecasting error. The x-axis indicates the parameter count on a logarithmic scale. The y-axis shows the out-of-sample forecasting MSE. CDP and CDP (Modulated) maintain their performance at the sparse end of the frontier, achieving competitive forecasting accuracy with far fewer parameters than deep learning baselines. The continuous and structured baselines (NOMP, SFW, MCA, Analysis K-SVD) are included, with Analysis K-SVD lying outside the displayed range (median test error above

28

, see subsection A.8 ).

Figure 8: Mean absolute SHAP value per atom index in greedy selection order, averaged over all experiments and datasets (log scale). Shading denotes 95% bootstrap confidence intervals. SFW is excluded from the figure because its per-atom attributions diverge at larger budgets (see text).

Method	Counted parameters
CDP / CDP (Mod.)	Every continuously optimized scalar: $3$ – $5$ atom-specific parameters plus one bias per atom
OMP	Frequency index and linear coefficient per selected dictionary atom
NOMP	Refined continuous frequency plus complex amplitude per sinusoid
SFW	Continuous atom position plus weight per atom
MCA	Selected linear coefficients over the grid-discretized sub-dictionaries
Analysis K-SVD	All entries of the learned analysis operator rows

Appendix

Table 5: Parameter-count definitions used in all Pareto analyses. Each method’s count comprises exactly the scalars required to reproduce its output.

Figure 9: Training and inference cost versus model complexity. (a) Fitting time in seconds (log–log scale). (b) Evaluation (inference) time in seconds. Shading denotes 95% bootstrap confidence intervals over datasets.

Method	$K{=}1$	$K{=}2$	$K{=}3$	$K{=}5$	$K{=}10$	$K{=}20$	$K{=}30$	$K{=}40$
CDP	12	20	29	63	129	223	329	417
NOMP	22	44	66	125	441	462	695	1722
SFW	46	91	141	216	451	858	1174	1879
MCA	1	1	1	2	1	1	1	1
Analysis K-SVD	95	86	60	73	76	46	30	14

Appendix

Table 6: Median fitting time in seconds per atom budget

K

for CDP and the continuous and structured baselines, taken over 13 datasets and 10 seeds.

Hyperparameter	Value	Description
scout_batch_size	10	Number of candidate atoms evaluated in parallel during truncated optimization
scout_epochs	5	Number of L-BFGS steps per candidate during the truncated scouting phase
global_epochs	100	Number of L-BFGS steps during full optimization of the selected atom
num_global_optims	5	Number of joint global refinement passes over all active atoms
n_atoms	{1,2,3,5,8,10,15,20,30,40}	Atom budgets evaluated for the Pareto frontier
atom_types (CDP)	All except ModulatedAtom	Atom families available for greedy selection

Appendix

Table 7: CDP hyperparameters used in all experiments. CDP uses all atom types excluding the ModulatedAtom, while CDP (Modulated) includes it.

Figure 10: Electricity. Reconstruction (a) and forecasting (b) MSE versus parameter count.

Figure 11: ETTh1. Reconstruction (a) and forecasting (b) MSE versus parameter count.

Figure 12: ETTh2. Reconstruction (a) and forecasting (b) MSE versus parameter count.

Figure 13: ETTm1. Reconstruction (a) and forecasting (b) MSE versus parameter count.

Figure 14: ETTm2. Reconstruction (a) and forecasting (b) MSE versus parameter count.

Figure 15: Exchange. Reconstruction (a) and forecasting (b) MSE versus parameter count.

Figure 16: PEMS03. Reconstruction (a) and forecasting (b) MSE versus parameter count.

Figure 17: PEMS04. Reconstruction (a) and forecasting (b) MSE versus parameter count.

Figure 18: PEMS07. Reconstruction (a) and forecasting (b) MSE versus parameter count.

Figure 19: PEMS08. Reconstruction (a) and forecasting (b) MSE versus parameter count.

Figure 20: Solar. Reconstruction (a) and forecasting (b) MSE versus parameter count.

Figure 21: Traffic. Reconstruction (a) and forecasting (b) MSE versus parameter count.

Figure 22: Weather. Reconstruction (a) and forecasting (b) MSE versus parameter count.

Time Series Decomposition

Time series decomposition separates evolving signals into components such as trend, seasonality, cycles, and residual variation.

See all 21 papers

Multi-Dictionary Learning for Low Rank Sparse Coding
Sep 12, 2025Boya Ma, Abram Magner, Maxwell McNeil +1Representation Learning Sample Complexity
University at Albany – SUNY, Computer Science, Albany, NY, USA
Variational decomposition autoencoding improves disentanglement of latent representations
Jan 11, 2026Ioannis N. Ziogas, Aamna Al Shehhi, Ahsan H. Khandoker +1Disentangled Representation Learning Variational Autoencoders
Dept. of Biomedical Engineering, Khalifa University of Science and Technology, Abu Dhabi, UAE · School of Electrical and Computer Engineering, Aristotle University of Thessaloniki, Thessaloniki, Greece
Inference and learning in sparse autoencoders as natural gradient flow
Oct 5, 2026Hadi Vafaii, Tejas Rao, David Chanin +6Gradient Descent Sparse Autoencoders
Astera Institute · UC Berkeley · UCL +2

Sep 12, 2025J/K move · Enter open · S save

Multi-Dictionary Learning for Low Rank Sparse Coding

Boya Ma, Abram Magner, Maxwell McNeil, Petko Bogdanov

University at Albany – SUNY, Computer Science, Albany, NY, USA

Representation Learning Sample Complexity Dictionary Learning Sparse Coding

Open paper AskPDF arXiv

Sparse dictionary coding represents signals as linear combinations of a few dictionary atoms. It has been applied to images, time series, graph signals and multi-way spatio-temporal data by jointly employing temporal and spatial dictionaries. Data-agnostic analytical dictionaries, such as the discrete Fourier transform, wavelets and graph Fourier, have seen wide adoption due to efficient implementations and good practical performance. On the other hand, dictionaries learned from data offer sparser and more accurate solutions but require learning of both the dictionaries and the coding coefficients. This becomes especially challenging for multi-dictionary scenarios since encoding coefficients correspond to all atom combinations from the dictionaries. To address this challenge, we propose a low-rank coding model for 2-dictionary scenarios and study its data complexity. Namely, we establish upper and lower bounds on the number of samples needed to learn dictionaries that generalize to unseen samples from the same distribution. We propose an alternating convex optimization solution, called AODL, which employs alternating optimization between the sparse coding matrices and the learned dictionaries. We demonstrate its quality for data reconstruction and missing value imputation in both synthetic and real-world datasets. For a fixed reconstruction quality, AODL learns up to

90\%

sparser solutions compared to non-low-rank and analytical (fixed) dictionary baselines. In addition, the learned dictionaries reveal interpretable insights into patterns from training samples.

Abstract

Time Series Decomposition

Explore similar work

Multi-Dictionary Learning for Low Rank Sparse Coding

Variational decomposition autoencoding improves disentanglement of latent representations

Inference and learning in sparse autoencoders as natural gradient flow