We present a hypernetwork that generates the parameters of an implicit neural representation (INR) for a given signal in a single forward pass, without test-time optimization. Existing hypernetworks for this problem either ignore the chain-rule structure of the test-time optimization that produces an INR or, when they preserve it, instantiate the structure with MLPs whose parameter count grows quartically in target-network width. Our central contribution is to show that this parameter-count cost is structural to the MLP instantiation, not to the optimization-oriented principle of preserving chain-rule structure, and to resolve it by replacing the MLP body with attention. The resulting hypernetwork combines three design choices: an output-neuron-centric tokenization that aligns the token axis with each layer's output-neuron axis; an information-flow consistency rule that fixes the role of every cross-attention module from the representation its output should occupy; and a token-space iterative rollout that realizes the five quantities one optimization step on the target INR computes as five cross-attention modules with parameter count independent of target-network width. Across five 2D and 3D INR generation benchmarks, our method outperforms the state-of-the-art single-pass and optimization-oriented baselines; scales to target-network widths at which the MLP-based optimization-oriented paradigm exhausts GPU memory; and produces the highest result on NeRF generation without ground-truth INR weights supervision.
Figures & tables
Figure 1: One iterative module of the proposed hypernetwork HΩ , illustrated for a three-layer MLP target INR ( L=3 ); the construction extends to arbitrary L . At iteration t , the forward chain (left, modules ϕi ) produces the pseudo-activation tokens τit from the parameter tokens ωi(t) , seeded by τ0t from zζ through the Cross-Attention (CA) block. The backward block (right) realizes one chain-rule step at each layer: ψiτ and ψiω produce the local Jacobian tokens Ji−1τ,t and Jiω,t , which ψi−1δ and ψi∇ contract with δit to produce the propagated loss-signal δi−1t and the parameter-gradient tokens ∇ωiL ; the backward recursion is seeded by δLt , formed from τLt and zζ via the concat-plus-CA module. The ADD block then applies the refinement with predicted step size and forwards to the next module.
Figure 2: Two tokenization schemes for the layer- i weight matrix Wi . Left (ours): the output-neuron-centric scheme groups each row of Wi into one token ωi,j , placing each output neuron’s parameters into a single, co-adapted unit. Right: the input-neuron-centric scheme groups each column of Wi into one token, collecting parameters that originate from one input neuron to multiple output neurons that may encode unrelated features and are therefore not co-adapted.
Dataset
Metric
VAMoH
HyPoGen
LDMI
Ours
CelebA-HQ
PSNR ↑
21.3
16.2
24.8
27.7
SSIM ↑
0.6303
0.4550
0.6947
0.8635
LPIPS ↓
0.1332
0.1463
0.0556
0.0296
ERA5
PSNR ↑
39.0
40.0
44.6
49.3
SSIM ↑
0.8090
0.9947
0.9952
0.9991
LPIPS ↓
0.1663
0.0008
0.0003
0.0001
Table 1: Quantitative comparison on 2D INR reconstruction on CelebA-HQ, ERA5, and ImageNet.
Figure 3: Visual comparison of generated INRs across four datasets. Left: 2D INR generation on (a) CelebA-HQ and 3D shape reconstruction on (b) ShapeNet Chairs. Right: 2D INR generation on (a) ImageNet and novel view synthesis on (b) Pre-trained NeRF. Columns show HyPoGen, LDMI, Ours, and the ground truth (GT). Qualitative results on ERA5, as well as additional samples for all four datasets shown here, are provided in Appendix 6.8.5 .
Dataset
Metric
VAMoH
HyPoGen
LDMI
Ours
ShapeNet Chairs
Accuracy ↑
96.8%
95.2%
97.3%
97.9%
PSNR ↑
15.6
19.0
22.4
23.8
Pre-trained NeRF
Metric
HyperDiff.
HyPoGen
LDMI
Ours
Weights only
PSNR ↑
20.0
19.9
19.3
22.0
Images only
PSNR ↑
N/A
24.9
26.2
28.8
Weights & Images
PSNR ↑
17.5
25.5
26.5
27.8
Table 2: Quantitative comparison on 3D INR generation. 3D shape reconstruction on ShapeNet Chairs (top) and novel view synthesis on Pre-trained NeRF (bottom) under three supervision settings. “HyperDiff.” denotes HyperDiffusion.
Params (M)
GPU Memory (GB)
Infer Time @ b=16 (ms)
Dataset
HyPoGen
LDMI
Ours
HyPoGen
LDMI
Ours
HyPoGen
LDMI
Ours
CelebA-HQ
128.8
40.5
41.2
0.72
0.35
0.42
6.6
17.2
24.5
ImageNet
578.7
544.9
124.5
8.20
8.12
5.51
66.8
84.5
58.5
ERA5
93.1
48.9
21.3
0.59
0.51
0.33
21.2
25.0
24.6
Shapes3D
8.3
30.9
4.1
0.82
0.79
0.80
4.7
8.3
10.2
Pretrained NeRF
1049.9
515.5
482.5
14.51
12.47
2.51
2382
2541
2018
Table 3: Computational efficiency evaluated at batchsize 16 on each dataset.
Variant
CelebA-HQ
ImageNet
NeRF
Ours (Full)
27.7
22.9
27.8
Column-wise Tokenization
25.3
19.8
25.1
Reversed Q-KV
23.5
18.3
22.9
Table 4: Ablations on the two architectural principles of Sec. 3 . PSNR ( ↑ ) on CelebA-HQ, ImageNet, and Pre-trained NeRF ( Weights & Images ).
Figure 4: Empirical scalability analysis. (a) Performance versus training-data ratio on NeRF generation, testing how accuracy improves with data scale. (b) Parameter count and PSNR versus target-INR width on CelebA-HQ at batch size 64, testing the parameter-count prediction of Eq. 16 : HyPoGen’s parameter count grows rapidly while ours grows mildly with much more gains in PSNR.
Original dimensions
Under compression
HyPoGen (MLP body)
O(ni2⋅ni−12)
O(ni2⋅d2)
Ours (attention body)
O(ni−12)
O(d2)
Structural difference
factor of ni2
factor of ni2
Table 5: Parameter count of the core iterative body per target-network layer per refinement step, for HyPoGen’s MLP-based core and our attention-based core, in the original target-network dimensions and under row-level compression to embedding dimension d . Encoders and decoders are shared between the two architectures and excluded from the count.
Standard layer
Cross-attention module ϕi
Linear mixing
Wihi−1
ωiτi−1⊤/d
Non-linearity
activation σ
row-wise softmax
Output-unit alignment
j -th row →j -th neuron
j -th query →j -th output token
Table 6: Component-by-component correspondence between a standard INR layer (Eq. 3 ) and the cross-attention module ϕi (Eq. 6 ) that realizes the forward block.
Module
Surrogate of
Output representation
V=K
Q
ψiτ
∂hi/∂hi−1
layer- (i−1) activation
τi−1
ωi
ψiω
∂hi/∂Wi
layer- i parameter
ωi
τi−1
ψi−1δ
δi⋅∂hi/∂hi−1
layer- (i−1) activation
Ji−1τ
δi
ψi∇
δi⋅∂hi/∂Wi
layer- i parameter
Jiω
δi
Table 7: QKV assignments of the four backward modules, derived from information-flow consistency given the output representation each module’s target-network counterpart lives in.
Target-INR width
128
384
512
768
1024
LDMI (original)
22.1
–
–
25.3
–
LDMI + Adaptive Scaling
20.4
22.8
25.2
25.3
26.1
Table 8: Width-scaling stability of LDMI with and without our adaptive scaling sub-module on CelebA-HQ at batch size 64, with SIREN as the target INR. PSNR (dB) is reported; “–” marks runs that diverge at iteration zero because the decoded Wi0 exits the SIREN-safe regime. The original LDMI fails on three of the five widths in the sweep, while LDMI augmented with SW(i) from Eq. 23 trains across the full range.
CelebA-HQ 64 × 64
ImageNet
ERA5
Chairs
Pre-trained NeRF
Learning rate
5e-5
5e-5
5e-5
5e-5
5e-5
Optimizer
Adam
Adam
Adam
Adam
Adam
Mini-batch size
64
128
128
32
96
Task Tokenizer dim
128
256
128
64
768
Each Block Hidden dim
128
768
128
64
768
Transformer layers
1
1
1
1
1
Table 9: Model hyperparameter across different datasets.
Dataset
Encoder
CelebA-HQ
2D Conv VAE
ERA5
2D Conv VAE
ImageNet
VQ-VAE (pre-trained, frozen)
ShapeNet Chairs
3D Conv VAE
Pre-trained NeRF
DINO ViT-B/16 + adaLN-Zero pose cond.
Table 10: Encoder used for each dataset.
Figure 5: Additional visual comparison on CelebA-HQ.
Figure 6: Additional visual comparison on ERA5.
Figure 7: Additional visual comparison on ImageNet.
Figure 8: Additional visual comparison on ShapeNet Chairs.
Figure 9: Additional visual comparison on Pre-trained NeRF.
Supervision
Metric
Trans-INR
GINR-IPC
Ours
Weights only
PSNR ↑
16.5
17.1
22.0
Images only
PSNR ↑
21.9
26.1
28.8
Weights & Images
PSNR ↑
16.6
18.2
27.8
SSIM ↑
0.8659
0.8899
0.948
LPIPS ↓
0.2618
0.1855
0.0742
Table 11: Additional NeRF-specific baselines on the Pre-trained NeRF dataset under three supervision settings. Our method outperforms both Trans-INR and GINR-IPC across all settings.
CelebA-HQ
ImageNet
NeRF
Variant
Params
PSNR
Params
PSNR
Params
PSNR
Full model
41.2
27.7
124.5
22.9
482.5
27.8
w/o Forward module
33.1
24.9
101.2
18.5
441.9
24.6
w/o Backward module
38.4
24.4
110.1
18.1
456.8
24.2
Table 12: Ablations on the structural blocks of the iterative module of Sec. 3.4 . Each variant disables one structural block—the forward chain, the role-specific backward decomposition, while keeping all other components and total compute matched to the full model. PSNR ( ↑ ) on CelebA-HQ, ImageNet, and Pre-trained NeRF ( Weights & Images ).
Variant
CelebA-HQ
ImageNet
NeRF
Ours (Full, T=3 )
27.7
22.9
27.8
Single Layer Optimization ( T=1 )
27.4
22.1
26.7
Table 13: Ablations on the structural blocks of the iterative module of Sec. 3.4 . PSNR ( ↑ ) on CelebA-HQ, ImageNet, and Pre-trained NeRF ( Weights & Images ).
Dataset
Seed 444
Seed 555
Seed 777
Seed 888
Seed 999
Mean ± Std
Pre-trained NeRF
27.7
27.8
27.7
27.8
27.8
27.76 ± 0.05
CelebA-HQ
27.8
27.7
27.7
27.5
27.6
27.66 ± 0.10
ERA5
48.9
49.3
49.4
49.2
49.5
49.26 ± 0.23
Table 14: PSNR stability across random seeds on multiple datasets. Each row reports our method’s PSNR under five independent training runs that differ only in the random seed used for initialization, together with the mean and standard deviation. NeRF results are reported under the Weights & Images supervision setting.
ImageNet
Pre-trained NeRF
Metric
Token Comp.
Layer Fus.
Ours
Token Comp.
Layer Fus.
Ours
PSNR (dB)
22.7
22.6
22.9
29.0 †
29.2 †
28.8
Infer Time @ b=16 (ms)
54.1
41.2
58.5
2098
2135
2018
Params (M)
132.82
99.43
124.5
484.39
491.43
482.5
Table 15: Efficiency comparison (batch size: 16) under two practical engineering strategies for scaling to wider or deeper target networks. NeRF results are reported under the Images only supervision setting.