Mapping Networks

Lord Sen; Shyamapada Mukherjee

cs.CVFeb 22, 2026

Mapping Networks

Authors: Lord Sen, Shyamapada Mukherjee

Organizations: National Institute of Technology Rourkela, Odisha, India

Abstract

The escalating parameter counts in modern deep learning models pose a fundamental challenge to efficient training and resolution of overfitting. We address this by introducing the Mapping Networks which replace the high dimensional weight space by a compact, trainable latent vector based on the hypothesis that the trained parameters of large networks reside on smooth, low-dimensional manifolds. Henceforth, the Mapping Theorem enforced by a dedicated Mapping Loss, shows the existence of a mapping from this latent space to the target weight space both theoretically and in practice. Mapping Networks significantly reduce overfitting and achieve comparable to better performance than target network across complex vision and sequence tasks, including Image Classification, Deepfake Detection etc, with $\mathbf{99.5\%}$ , i.e., around $500\times$ reduction in trainable parameters.

Figures & tables

Figure 1: State of the Existing Works and Ours in this field

Panel of Figure 2 — Figure 2: Parameter update snapshots showing distinct parameter manifolds in CNN evolution.

Figure 3: General Architecture for Mapping Networks.

Figure 4: Process of modulation of Mapping weights and training of latent vector z from epoch p to p+1.

Panel of Figure 5 — Figure 5: Training strategies used for Mapping Network

Method	# Params	MNIST	FMNIST
CNN1	537,994	99.32%	92.89%
Ours*	1024	98.78%	93.02%
Ours*	2072	99.56%	93.91%
Ours $\dagger$	4078	99.67%	94.83%
CNN2	108,618	98.69%	90.40%
Ours*	1024	97.88%	89.49%

Table 1: Image Classification with Mapping CNN

Method	# Params	Celeb-DF	FF++
CNN1	537,994	83.13%	82.44%
Ours*	1024	83.92%	81.11%
Ours*	2048	88.88%	85.23%
Ours $\dagger$	1956	88.78%	86.23%
Ours $\dagger$	2792	89.98%	88.05%
CNN2	108,618	79.03%	79.85%

Table 2: Mapping CNN on Deepfake Detection

Method	# Total	Pixel Acc	Loss	mIoU
CNN3	1,734,803	93.21%	0.1506	0.4957
Ours*	8192	97.92%	0.1233	0.4623
Ours $\dagger$	9126	97.56%	0.1002	0.4823

Table 3: Results on Image Segmentation

Method	# Params	MSE Loss
LSTM	12961	0.0035
Ours*	64	0.0019
Ours*	2048	0.00061

Table 4: Mapping LSTM on Air Pollution dataset

Method	# Params	Layers	CDF	FF++
ResNet50	25M	All	95.23%	91.78%
Ours*	2048	All	95.10%	91.02%
ResNet50	17M	L-4, FC	91.11%	88.03%
Ours*	1024	L-4, FC	92.10%	89.23%

Table 5: Fine Tuning ResNet50 via Mapping Networks

Method	# Params	Task Loss	+ Stab	+ Smooth	+ Alli	+ Sm + Ali	+ Stab + Sm	Full
CNN2	108,618	90.40%	-	-	-	-	-	-
Ours*	1024	87.79%	88.30%	88.85%	88.21%	88.66%	88.43%	89.49%
Ours*	2048	87.88%	89.91%	90.23%	90.11%	89.86%	90.67%	91.88%
Ours $\dagger$	1872	89.11%	89.56%	89.43%	89.32%	90.47%	91.11%	92.84%
Ours $\dagger$	2688	91.11%	91.89%	91.50%	91.67%	92.90%	93.63%	94.08%

Table 6: Ablation of Mapping Loss on FashionMNIST dataset

Method	# Params	MNIST	FMNIST
CNN2	108,618	98.69%	90.40%
Full DNN	6,753,104	97.12 %	90.11%
Ours*– WM	1024	95.62%	86.51%
Ours*– WM	2048	96.55%	87.66%
Ours*	1024	97.88%	89.49%
Ours*	2048	98.66%	91.88%

Table 7: Robustness Study of Mapping CNN

Method	# Params	MNIST	FMNIST
CNN2	108,618	98.69%	90.40%
CNN2 + LRD	35,914	98.12%	89.67%
CNN2 + Prune	10862	95.87%	87.91%
Ours*	2048	98.66%	91.88%
Ours* + LRD	1456	97.80%	90.67%
Ours*+ Prune	2048	95.93%	88.70%

Table 8: Imapct of Add Ons on Mapping Network

Weight-Space Learning

See all 17 papers

Predicting When Random Low-Dimensional Reparameterizations Train Neural Networks
Aug 12, 2026Andrew Cheng, Ali Eslamian, Jie Cheng +2Subspace Optimization
Department of Computer Science, Tsinghua University, Beijing, China · Department of Computer Science, The University of Manchester, Manchester, UK · Department of Computer Science, University of Kentucky, Lexington, KY, USA +3
Kilobyte Models: Neural Networks as a Seed and a Quantized Latent
Aug 1, 2026Sahil Rajesh DhayalkarNeural Network Compression
Arizona State University
Most ReLU Networks Admit Identifiable Parameters
May 5, 2026Moritz Grillo, Guido MontúfarRepresentation Identifiability Parameter Identifiability
Max Planck Institute for Mathematics in the Sciences, Leipzig · Departments of Mathematics and Statistics & Data Science, UCLA

Aug 12, 2026J/K move · Enter open · S save

Predicting When Random Low-Dimensional Reparameterizations Train Neural Networks

Andrew Cheng, Ali Eslamian, Jie Cheng, Mehdi Zargham+1

Department of Computer Science, Tsinghua University, Beijing, China · Department of Computer Science, The University of Manchester, Manchester, UK · Department of Computer Science, University of Kentucky, Lexington, KY, USA+3

Subspace Optimization

Open paper AskPDF arXiv

Neural networks can often be trained or fine-tuned through random low-dimensional reparameterization, where a small latent vector is mapped into a full parameter update by a frozen random map. This raises a practical question: how large must the latent search space be to reach a low-loss region? We first express the known accessibility transition in an equivalent conic form, centered for compact convex targets at the statistical dimension of the polar cone. Our main theoretical contribution is an orientation-resolved quadratic master formula that predicts the random-slice residual from both the curvature spectrum and the reference-to-solution displacement profile. It yields a self-consistent isotropic-orientation predictor and, in a conservative radius-only specialization, recovers the earlier Gaussian-width quadratic bound. Building on this analysis, we introduce Random Mapping Networks (RaMaN), which instantiate the predicted latent dimension using structured Hadamard or seed-regenerated Gaussian maps. These constructions avoid the O(dP) storage of dense random maps and reduce optimizer-state memory from O(P) to O(d). We also develop matrix-free curvature approximations and sweep-free dimension selection. Across controlled quadratic and neural-curvature experiments, the orientation-resolved predictor closely tracks measured transition locations and outperforms orientation-agnostic approximations when displacement direction matters. End-to-end experiments further show sharp, protocol-dependent training transitions across image and language models.

Abstract

Weight-Space Learning

Explore similar work

Predicting When Random Low-Dimensional Reparameterizations Train Neural Networks

Kilobyte Models: Neural Networks as a Seed and a Quantized Latent

Most ReLU Networks Admit Identifiable Parameters