Progressive Permutation Alignment with Temperature-Controlled Spectral Sharpening

Yuta Horioka; Naoya Chiba; Yuki Uranishi

cs.LGOct 21, 2026

Progressive Permutation Alignment with Temperature-Controlled Spectral Sharpening

Authors: Yuta Horioka, Naoya Chiba, Yuki Uranishi

Organizations: The University of Osaka

Abstract

In implicit neural representations (INRs), 3D signals are encoded directly in network weights, making stable weight-space operations essential. However, permutation symmetry causes independently trained yet functionally equivalent models to appear distant in weight space, often inducing large loss barriers under linear interpolation. Existing neuron alignment methods either commit early to discrete matches and become trapped in inconsistent local optima or rely on computationally expensive gradient optimization over continuous relaxations, both of which are unreliable in challenging cases such as NeRF. We adopt progressive alignment, a layer-wise strategy that follows inter-layer dependencies from input to output. At each layer, a correlation matrix is estimated and aligned with a discrete anchor obtained by the Hungarian algorithm, after which temperature-controlled spectral annealing gradually sharpens the correlation structure toward a permutation matrix. This update is deterministic and lightweight, avoiding the inner-loop gradient optimization required by continuous relaxation methods. A final refinement stage further optimizes the network layer-by-layer via weight matching, starting from the aligned solution. Experiments on MLPs, CNNs, and Neural Radiance Fields (NeRFs) show that the proposed method reliably recovers neuron correspondences and achieves stable alignment across architectures. In practical experiments aligning independently trained NeRF models, our method reduces the loss barriers under weight interpolation compared with existing approaches.

Figures & tables

Figure 1: Overview of Progressive Alignment with Temperature-Controlled Spectral Sharpening. Two independently trained neural fields, $\mathcal{N}_{A}$ and $\mathcal{N}_{B}$ exhibit permutation symmetry across the hidden layers. Our method aligns them in a progressive, layer-wise manner following the inter-layer dependency from input to output. At each layer, a correlation matrix is computed and gradually sharpened via temperature-controlled spectral projection.

Figure 2: Temperature-Controlled Spectral Sharpening for Layer-wise Permutation Update. At layer $l$ , given weights $W_{l}^{(A)}$ and $W_{l}^{(B)}$ , a correlation matrix $C_{l}$ that captures the neuron-wise similarity under fixed neighboring permutations is computed. The Hungarian algorithm first produces an anchor permutation $\hat{P}_{l}$ that provides a discrete reference alignment. Temperature-controlled spectral sharpening is applied to the aligned correlation matrix $\hat{P}_{l}^{\top}C_{l}$ , where spectral values are exponentiated by a temperature parameter $\tau$ .

Algorithm	MLP5		VGG11		ResNet18
Algorithm	$L_{1}$ Dist.	Precision	$L_{1}$ Dist.	Precision	$L_{1}$ Dist.	Precision
Weight Matching	$0.000$ $\pm 0.000$	$100.0$ $\pm 0.000$	$0.000$ $\pm 0.000$	$100.0$ $\pm 0.000$	$0.000$ $\pm 0.000$	$100.0$ $\pm 0.000$
Progressive Weight Matching	$0.000$ $\pm 0.000$	$100.0$ $\pm 0.000$	$0.000$ $\pm 0.000$	$100.0$ $\pm 0.000$	$0.000$ $\pm 0.000$	$100.0$ $\pm 0.000$
OT4P ( $\tau=0.7$ )	$0.000$ $\pm 0.000$	$100.0$ $\pm 0.000$	$6.223$ $\pm 0.271$	$99.74$ $\pm 0.033$	$6.376$ $\pm 0.298$	$99.29$ $\pm 0.090$
OT4P ( $\tau=0.5$ )	$0.000$ $\pm 0.000$	$100.0$ $\pm 0.000$	$7.075$ $\pm 0.201$	$99.64$ $\pm 0.036$	$5.768$ $\pm 0.273$	$99.45$ $\pm 0.065$
OT4P ( $\tau=0.3$ )	$0.000$ $\pm 0.000$	$100.0$ $\pm 0.000$	$6.746$ $\pm 0.225$	$99.74$ $\pm 0.029$	$5.820$ $\pm 0.611$	$99.42$ $\pm 0.135$

Table 1:

L_{1}

distance (converted by

\log(1+x)

) and Precision (%) of alignment algorithms across standard architectures (MLP, CNN)

Algorithm	$L_{1}$ Dist.	Precision
Weight Matching	0.000	100.0
Progressive Weight Matching	0.000	100.0
OT4P ( $\tau=0.3$ )	12.76	14.55
OT4P ( $\tau=0.5$ )	6.818	99.16
OT4P ( $\tau=0.7$ )	6.720	99.21
Progressive OT4P ( $\tau=0.3$ )	12.92	1.455

Table 2:

L_{1}

distance (converted by

\log(1+x)

) and Precision (%) on NeRF

Method	RMSE $\downarrow$	PSNR $\uparrow$
Naive	0.491	6.205
Weight Matching	0.294	11.07
Progressive Weight Matching	0.306	10.64
OT4P ( $\tau=0.3$ )	0.489	6.239
OT4P ( $\tau=0.5$ )	0.495	6.118
OT4P ( $\tau=0.7$ )	0.494	6.160

Table 3: Quantitative comparison of interpolated aligned models at

\lambda=0.5

Panel of Figure 3 — Figure 3: Example of interpolation results for different $\lambda$ values. (a) Naive, (b) Weight Matching, (c) Progressive Weight Matching, (d) OT4P, (e) Progressive OT4P, (f) Proposed (w/o weight matching refinement), and (g) Proposed.

Panel of Figure 4 — Figure 4: Quantitative comparison across different metrics.

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Category	RMSE $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$
AM	Data-dependent	0.161	16.22	0.775
STE	Weight-based	0.268	11.84	0.661
Ours	Weight-based	0.199	14.40	0.710

Appendix

Table S1 : Quantitative comparison against Activation Matching (AM) and STE at the interpolation midpoint (

\lambda=0.5

). Note that AM is a data-dependent method, whereas STE and our proposed method are strictly weight-based (activation-free).

Method	RMSE $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$
Naive	0.491	6.205	0.627
Weight Matching	0.294	11.07	0.653
Progressive Weight Matching	0.306	10.64	0.637
Ours (w/o SVD, w/o Ref.)	0.264	12.39	0.674
Ours (w/o SVD)	0.248	12.70	0.669
Ours (w/o Ref.)	0.199	14.22	0.697

Appendix

Table S2 : Quantitative ablation results on the importance of Spectral Projection and Sequential Layer-wise Refinement. Average metrics across the NeRF Synthetic dataset are reported at the midpoint of interpolation (

\lambda=0.5

Figure S1 : Quantitative evaluation of interpolation trajectories for different ablation variants across $\lambda\in[0,1]$ . The integration of spectral projection significantly elevates the overall performance curve, while the addition of sequential layer-wise refinement further stabilizes the alignment, achieving the highest quality at the midpoint ( $\lambda=0.5$ ).

Schedule	RMSE $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$
Naive	0.491	6.205	0.616
Weight Matching	0.294	11.07	0.621
Progressive Weight Matching	0.306	10.64	0.601
Two-step ( $\tau\in\{1.0,0.0\}$ ) w/o Ref.	0.241	12.89	0.698
Two-step ( $\tau\in\{1.0,0.0\}$ )	0.256	12.85	0.697
Three-step ( $\tau\in\{1.0,0.5,0.0\}$ ) w/o Ref.	0.199	14.22	0.697

Appendix

Table S3 : Quantitative ablation results on the temperature scheduling of

\tau

. We evaluate different schedule granularities both with and without the final refinement. Average metrics across the NeRF Synthetic dataset are reported at

\lambda=0.5

Figure S2 : Impact of temperature scheduling on interpolation quality across different $\lambda$ values. The three-step schedule strikes the optimal balance between continuous exploration and discrete exploitation, outperforming both the abrupt binary hard-switch (two-step) and the overly gradual approach (five-step).

Algorithm	MLP5	VGG11	ResNet18
Weight Matching	0.538 $\pm$ 0.080	169.5 $\pm$ 14.06	7.937 $\pm$ 0.501
Progressive Weight Matching	0.052 $\pm$ 0.047	10.43 $\pm$ 0.064	0.752 $\pm$ 0.091
OT4P ( $\tau=0.7$ )	15.54 $\pm$ 1.574	2307 $\pm$ 12.17	54.99 $\pm$ 5.747
OT4P ( $\tau=0.5$ )	16.36 $\pm$ 0.358	2605 $\pm$ 591.2	53.06 $\pm$ 0.097
OT4P ( $\tau=0.3$ )	17.24 $\pm$ 0.850	2380 $\pm$ 19.86	55.64 $\pm$ 5.661
Progressive OT4P ( $\tau=0.7$ )	10.24 $\pm$ 0.626	2059 $\pm$ 219.8	46.66 $\pm$ 4.084

Appendix

Table S4 : Alignment computation time (mean and standard deviation, [sec])

Algorithm	NeRF (Shuffled Weights)	NeRF (Different Initializations)
Weight Matching	15.60 $\pm$ 0.871	153.7 $\pm$ 25.09
Progressive Weight Matching	1.266 $\pm$ 0.097	14.55 $\pm$ 2.430
OT4P ( $\tau=0.3$ )	48.40 $\pm$ 23.75	38.72 $\pm$ 11.11
OT4P ( $\tau=0.5$ )	66.50 $\pm$ 5.688	17.05 $\pm$ 4.585
OT4P ( $\tau=0.7$ )	60.63 $\pm$ 7.621	66.06 $\pm$ 13.83
Progressive OT4P ( $\tau=0.3$ )	93.62 $\pm$ 3.201	92.21 $\pm$ 2.056

Appendix

Table S5 : Computation time for NeRF alignment (mean and variance, [sec])

Figure S3 : Average SSIM over the eight scenes of the NeRF Synthetic dataset. The horizontal axis denotes the interpolation coefficient $\lambda$ , and the vertical axis denotes the mean SSIM.

Panel of Figure 16 — Figure S4 : SSIM for each scene in the NeRF Synthetic dataset. The proposed method shows a smaller drop in SSIM in the intermediate region of interpolation.

Panel of Figure 17 — Figure S5 : RMSE along the linear interpolation path for all eight scenes of the NeRF Synthetic dataset. Lower is better.

Panel of Figure 18 — Figure S6 : PSNR along the linear interpolation path for all eight scenes of the NeRF Synthetic dataset. Higher is better.

Panel of Figure 19 — Figure S7 : Interpolation results at different values of $\lambda$ for a novel test view. (a) Naive, (b) Weight Matching, (c) Progressive Weight Matching, (d) OT4P, (e) Progressive OT4P, (f) Proposed w/o Ref, and (g) Proposed.

Panel of Figure 20 — Figure S8 : Example of interpolation results for different $\lambda$ values for the Chair scene. (a) Naive, (b) Weight Matching (c) Progressive Weight Matching, (d) OT4P, (e) Progressive OT4P, (f) Proposed w/o Ref, (g) Proposed.

Panel of Figure 21 — Figure S9 : Example of interpolation results for different $\lambda$ values for the Ficus scene. (a) Naive, (b) Weight Matching (c) Progressive Weight Matching, (d) OT4P, (e) Progressive OT4P, (f) Proposed w/o Ref, (g) Proposed.

Panel of Figure 22 — Figure S10 : Example of interpolation results for different $\lambda$ values for the Drums scene. (a) Naive, (b) Weight Matching (c) Progressive Weight Matching, (d) OT4P, (e) Progressive OT4P, (f) Proposed w/o Ref, (g) Proposed.

Panel of Figure 23 — Figure S11 : Example of interpolation results for different $\lambda$ values for the Hotdog scene. (a) Naive, (b) Weight Matching (c) Progressive Weight Matching, (d) OT4P, (e) Progressive OT4P, (f) Proposed w/o Ref, (g) Proposed.

Panel of Figure 24 — Figure S12 : Example of interpolation results for different $\lambda$ values for the Mic scene. (a) Naive, (b) Weight Matching (c) Progressive Weight Matching, (d) OT4P, (e) Progressive OT4P, (f) Proposed w/o Ref, (g) Proposed.

Panel of Figure 25 — Figure S13 : Example of interpolation results for different $\lambda$ values for the Materials scene. (a) Naive, (b) Weight Matching (c) Progressive Weight Matching, (d) OT4P, (e) Progressive OT4P, (f) Proposed w/o Ref, (g) Proposed.

Panel of Figure 26 — Figure S14 : Example of interpolation results for different $\lambda$ values for the Ship scene. (a) Naive, (b) Weight Matching (c) Progressive Weight Matching, (d) OT4P, (e) Progressive OT4P, (f) Proposed w/o Ref, (g) Proposed.

Implicit Neural Representations

Implicit neural representations encode signals and fields as continuous functions that can be queried at arbitrary coordinates and resolutions.

See all 188 papers

Permutation-Equivariant Flow Matching for Alignment-Free Neural Weight Generation
Sep 26, 2026Arkadi Piven, Yam Eitan, Guy Bar-Shalom +5Permutation-Equivariant Neural Networks Flow Matching
Technion – Israel Institute of Technology · Technical University of Munich · Munich Center for Machine Learning +1
Multi-Way Representation Alignment
Feb 5, 2026Akshit Achara, Tatiana Gaintseva, Mateo Mahaut +5Neural Representation Geometry Latent Space Alignment
King’s College London, London, United Kingdom · Queen Mary University of London, London, United Kingdom · Universitat Pompeu Fabra, Barcelona, Spain +5
Flag Varieties: A Geometric Framework for Deep Network Alignment
May 11, 2026Jingchuan Xiao, Xinyi Sui, Cihan RuanNeural Representation Geometry Neural Collapse
Department of Mathematics and Computer Studies, Mary Immaculate College, Ireland · Department of Computer Science and Engineering, Santa Clara University, USA

Sep 26, 2026J/K move · Enter open · S save

Permutation-Equivariant Flow Matching for Alignment-Free Neural Weight Generation

Arkadi Piven, Yam Eitan, Guy Bar-Shalom, Fabrizio Frasca+4

Technion – Israel Institute of Technology · Technical University of Munich · Munich Center for Machine Learning+1

Permutation-Equivariant Neural Networks Flow Matching Equivariant Neural Networks

Open paper AskPDF arXiv

A trained neural network can be represented by a parameter vector in high dimensions. Learning distributions over these vectors enables the generation of new models across various tasks and architectures. A central challenge is permutation symmetry: permuting hidden neurons can produce distant parameter vectors representing the same function. This introduces variations that a generative model must account for when learning from trained networks. Existing methods typically address this using networks derived from a common base model or costly approximate neuron alignment. We instead parameterize a flow-matching velocity field with a permutation-equivariant Graph Meta Network, enabling direct learning from independently trained networks without alignment. Extensive experiments show that our method closely reproduces the joint statistics of accuracy, functional similarity, and weight similarity of independently trained collections, providing evidence of generation beyond checkpoint memorization. A single conditional model also generates task-specific networks on heterogeneous architectures and generalizes to unseen hidden-width configurations. On a tabular domain-shift task, intermediate conditioning produces individual networks with performance comparable to logit ensembles across both domains. Taken together, our results show how permutation equivariance enables learning from diverse collections of independently trained networks without permutation alignment.

Abstract

Implicit Neural Representations

Explore similar work

Permutation-Equivariant Flow Matching for Alignment-Free Neural Weight Generation

Multi-Way Representation Alignment

Flag Varieties: A Geometric Framework for Deep Network Alignment