cs.CVMay 8, 2026

Is Class Signal Clustered or Routed in Task-Induced Implicit Neural Representation Weight Spaces?

Authors: Xinyi Guo, Mingyi He, Haobin Ding, Weiming Chen, Xinrui Chen, Jiawen Li, Di Zhang, Minxi Ouyang, +2 more

Organizations: South China Normal University · Beijing University of Chemical Technology · Tsinghua University · Xi’an Jiaotong University

Abstract

Implicit neural representations (INRs) encode images as neural-network weights, making image classification a problem of weight-space classifiability. A natural geometric hypothesis is that classifier feedback should make image-specific weights cluster by class in the shared-anchor coordinate. We test this hypothesis in the SIREN-based Meta Weight Transformer (MWT) regime, where end-to-end training meta-learns a shared initialization and inner-loop update schedule for fitting image-specific SIRENs. We find that this prediction fails. Exposed weight-space geometry and supervised clustering pressure do not reliably track trained-reader accuracy; clustering can even make local neighborhoods more class-consistent while making the trained reader worse. Crucially, the reader constructs rather than inherits class-aligned geometry: token-flow diagnostics show that class-aligned neighborhoods become strongly predictive of trained-reader accuracy only after late reader interactions, not in the input coordinate. We further identify the native SIREN bias column in the augmented weight token as a low-dimensional, sample-dependent causal readout route for the trained reader; targeted controls rule out generic scalar-column and marginal-distribution artifacts. The diagnosis motivates interventions that strengthen reader routing, add an explicit bias route, or use denser inner-loop fitting; under the lane-specific training conventions used here, route-directed variants often outperform the shared-anchor baseline but interact non-additively. Task-induced INR weights are classifiable not because they form raw geometric clusters, but because their class signal is routed through the reader.

Explore similar work

CardsList
Jul 31, 2026cs.CV

Weight-Space Mixture-of-Experts for Implicit Neural Representation Classification

Implicit Neural Representations (INRs) encode signals as the weights of a coordinate-based neural network and have recently been proposed as an alternative domain for downstream learning. While promising, classification directly in weight space remains challenging due to the high dimensionality and complex structure of INR parameters. Furthermore, the way discriminative information is distributed across INR weights remains poorly understood. We propose a hierarchical Mixture-of-Experts (HMoE) Transformer that processes INR weights using conditional computation aligned with the structure of the underlying implicit network. Coupled with a meta-learning framework that shapes INR parameters for downstream tasks, our model achieves state-of-the-art accuracy across standard benchmarks, ranging from low-resolution datasets to high-resolution ImageNet-1K. To gain insight into how INRs encode discriminative information, we develop weight-space attribution and pruning methods that identify parameters most relevant for classification. These analyses reveal how class-specific structure emerges within INR layers and support the suitability of MoE architectures for weight-space learning. Our approach advances both the performance and interpretability of weight-space classifiers.
Stanislaw Janik, Michal Byra
Implicit Neural RepresentationsWeight-Space
AskPDF
Jun 8, 2026cs.LG

Beyond Neural Collapse: Task-Intrinsic Geometry Governs Neural Representations in Modular Arithmetic

While neural collapse (NC) predicts that a KK-class-balanced classifier should organize terminal representations as a (K−1)(K-1)-dimensional simplex equiangular tight frame (ETF), modular addition consistently enters a different regime: networks compress to a two-dimensional cyclic geometry in which both classifier weights and token embeddings lie on circles. We refine the explanation of this phenomenon in three directions. First, we formalize a layerwise non-uniform training mechanism: downstream classifier weights are driven by dense cross-entropy gradients into a rank-2 equiangular configuration before upstream embeddings fully reorganize, and once this classifier plane forms, backpropagated feature gradients constrain embedding motion to the same plane while weight decay suppresses orthogonal components. Second, after this subspace locking, the induced in-plane dynamics admit an entropy-regularized transport interpretation on S1S^1; combined with modular-addition labels, this reduces embedding formation to phase alignment, whose minimizers are single-frequency characters of Z/PZ\mathbb{Z}/P\mathbb{Z} and hence equal-angle points on a circle. Third, we quantify why this solution prevails over NC: a simplex ETF gains only an O(1)O(1) advantage in cross-entropy, whereas the cyclic rank-2 solution enjoys a Θ(K)Θ(K) advantage under Schatten or weight-decay surrogates, yielding a critical threshold λcrit=Θ(1/K)λ_{\mathrm{crit}} = Θ(1/K). Our results explain both why classifier weights move first and why embeddings subsequently align with them, showing that grokking on modular arithmetic is governed not by maximal separation alone but by a task-structured trade-off between separation, symmetry, and complexity.
Hu Tan, Kuo Gai, Shihua Zhang
Neural CollapseNeural RepresentationsModular AdditionImplicit Neural RepresentationsHyperspherical Embedding GeometryDownstream Fine-Tuning
AskPDF
Jul 11, 2026cs.LG

Error Aware Distribution Prediction for Lightweight Implicit Neural Representations

Implicit neural representations (INRs) offer compact encoding of volumes, but as lossy approximators, inevitably have prediction errors. We consider INRs that can simultaneously encode relative error scales by predicting distributions using tools from uncertainty estimation. Typically, uncertainty estimation relies on computationally expensive approaches or on predefined parametric assumptions about the predictive distribution (e.g., Gaussian). In this study, we propose a lightweight method that reformulates regression-based INR training as a classification task by discretizing continuous targets into bins, enabling flexible distribution modeling to capture complex multimodal behaviors. We analyze the trade-off between regression and classification for INR training and demonstrate that the classification setting tends to achieve high reconstruction quality and competitive error awareness through uncertainty estimation, compared to regression-based approaches.
Zhimin Li, Jake D. Balla, Joshua A. Levine
Implicit Neural RepresentationsDistributional LearningRandomized Global-Local Density EstimationPosteriorReconstruction Quality
AskPDF