Weight-space programming fixes every parameter of a feed-forward network by hand, with no gradient descent anywhere in the construction. Every demonstration so far starts from a target function handed to the model as a rule system, making it a compilation problem. A supervised dataset withholds that specification, so the procedure must do the statistical inference itself and then express the result in the parameterisation it was given. We ask whether a large language model can do both. It is shown a small dataset in full, with the architecture of a feed-forward network, and writes out every weight and bias as a literal number, with no optimizer, no gradient, no code execution, and no chance to evaluate it first. Each construction is sealed and pre-registers its expected score. The graded suite runs from two toy problems, through nine synthetic problems disguised by a hidden rotation, to the classical benchmarks and one real engineering regression. The frozen network matches the closed-form optimum wherever that optimum is linear, reaches the irreducible error of the generating process on three rungs where it is not, and beats every fitted baseline on two of those, including the same architecture trained by gradient descent. It fails on two rungs, both times because the structure to be found is oblique to the coordinates it analysed. Three further constructors separate two failure modes a single score conflates: one returns no parameter vector at all on the hardest rungs, another delivers everywhere while writing networks worse than predicting the mean. One of those failures proves avoidable: another constructor solves the oblique-hinge rung by a margin that survives multiple-comparison correction. What predicts whether the hidden layer earns its keep is the training error of a linear separator, which the composite complexity index misses.
Figures & tables
Figure 1: The capability test this paper applies under many conditions. A language model is shown a complete structured dataset and the required shapes of a feed-forward network, and must write out, from reasoning alone, every entry of that network’s parameters so that its frozen forward pass predicts the target variable on held-out rows. Everything that varies across the paper, which dataset, whether that dataset is a classical one the model may have memorised or a synthetic one it cannot have seen, which architecture is being filled in, and which reference solution the result is compared against, is a controlled variation of this one test.
Figure 2: The suite ordered by measured difficulty, with the easiest rung at the bottom of each panel. The filled marker is the composite complexity index, the mean of the three open markers beside it, and every measure is normalised so that larger is harder and is computed from the training rows alone after the constructions on that rung were sealed. The two task types are drawn separately because their headline triples differ: for classification the training error of a linear separator, the fraction of points on a class boundary in a minimum spanning tree, and the leave-one-out error of the nearest-neighbour rule; for regression the mean squared residual of the least-squares fit, that residual on interpolated points, and the correlation deficit of the best single feature. The disagreement between a mean and its members is visible directly. On s3_xor the linear separator fails on a fifth of the rows while the nearest-neighbour rule fails on under one per cent, which places the rung low on the index and high on the only measure that predicts what a hidden layer is worth; on r4_sine all three members agree, and that rung is the hardest in the suite by every one of them.
Figure 3: The two toy rungs with the hand-written solutions drawn over them. Filled markers are the rows placed in the prompt and hollow markers the held-out rows, which the constructing model never saw. On the left the hand-written line and the least-squares line computed from the same twenty rows are separated by less than the width of the stroke. On the right the hand-written boundary separates the forty-eight specimens without error; the closest held-out row lies more than two within-group standard deviations from it.
Figure 4: The complete toy_line construction, which is the smallest instance in the paper of what every other rung does at scale. Nothing between the first box and the last was executed: the four sums were taken by reading the table, the two divisions were done in the model’s reasoning, and the two numbers that came out were written into a module that contains no import, no call and no name other than its own parameter arrays. The same module declares, before the seal, that it expects a coefficient of determination between 0.995 and 0.9995 ; it scored 0.9997 .
rung
task
n
d
L2
linear
MLP
best baseline
ceiling
s1_sep
classification
80/80
4
0.000
1.0000
1.0000
1.0000
1.0000
s2_lin
classification
100/100
4
0.090
0.8800
0.8800
0.9400
0.9000
s3_xor
classification
120/120
2
0.200
0.7583
1.0000
0.9917
0.9877
s4_ball
classification
120/120
2
0.367
0.6167
0.9917
1.0000
0.9896
s5_spiral
classification
120/120
2
0.375
0.6167
0.5750
0.9917
0.9997
r1_lin
regression
100/100
5
0.004
0.8872
0.8872
0.9117
0.9004
Table 1: The nine synthetic rungs. “Linear” and “MLP” are the hand-written constructions for the two architectures; “best baseline” is the strongest of the twelve fitted predictors on that rung; “ceiling” is one minus the irreducible error for classification and the attainable coefficient of determination for regression, both computed from the sealed generating process; “half-plane” is the accuracy of the best half-plane found by exhaustive search, reported for the two-dimensional rungs where the optimal boundary is not linear. “ L2 ” is the training error of a linear separator for classification and the mean squared residual of the least-squares fit for regression, computed from the training rows only and after every construction was sealed. Bold marks the three rungs on which a hand-written construction matches or beats every one of the twelve fitted baselines.
Figure 5: Decision boundaries of the three hand-written mlp8 constructions whose rungs are not linearly separable, drawn over all two hundred and forty rows of each, with the held-out accuracy in the title. The parity boundary is the exact pair of lines through the centre of the cluster parallelogram; the radial boundary is an octagon whose corners the sum of its own hinges has rounded to within one per cent of a circle; the spiral boundary is a set of parallel stripes in a tilted coordinate, which tracks the arms where they are far apart and inverts where they wind in.
Figure 6: What the hidden layer buys, against how far from linearly separable the training rows are. For the classification rungs the two quantities agree closely over the range in which the construction succeeds: the hand-written linear arm loses almost exactly the fraction of points that no half-plane can place correctly, and the hidden layer recovers almost exactly that fraction. The one classification point below the line is s5_spiral , where the hidden layer was spent and lost accuracy. The regression points sit far above the line because the residual of a least-squares fit on a target scaled to the unit interval is not on the same scale as a difference of coefficients of determination; their ordering, not their height, is the comparable quantity.
Figure 7: The real rung. On the left, the hand-written eight-unit network’s predictions against the measured compressive strength of the seven hundred and five held-out mixtures, with the identity drawn for reference; the spread is wider at the top of the range, which is where the multiplicative age effect the construction could not represent does the most damage. On the right, the piecewise-linear age curve the model derived from matched pairs of rows, evaluated at the median mixture and drawn on a logarithmic age axis, on which a logarithmic law would be a straight line.
rung
L2
ceiling
base
Opus 5
Grok 4.7
Sonnet 5
Haiku 4.5
toy_line
0.000
0.9994
0.9997
0.9997
0.9997
0.9997
0.9996
toy_gap
0.000
1.0000
1.0000
1.0000
1.0000
1.0000
1.0000
s1_sep
0.000
1.0000
1.0000
1.0000
1.0000
0.9750
0.9625
s2_lin
0.090
0.9000
0.9400
0.8800
0.9100
0.8800
0.7800
s3_xor
0.200
0.9877
0.9917
1.0000
0.9333
0.6333
0.6500
s4_ball
0.367
0.9896
1.0000
0.9917
0.9833
1.0000
0.5750
Table 2: The four constructor arms on all twelve rungs, each entry the better of that arm’s two architectures on the same held-out rows. “Ceiling” is one minus the irreducible error for classification and the attainable coefficient of determination for regression, both from the sealed generating process; the real rung has neither. “Base” is the strongest of the twelve fitted predictors. A dash marks a rung for which that constructor returned nothing across four dispatches at two output ceilings. The means at the foot are taken over the nine rungs every arm delivered, which is the only set on which the four are comparable with one another. Bold marks the highest entry of each row among the four arms, and every arm that ties it.
family
rung and architecture
Δ
p
in favour of
Opus against Grok
r2_kink , MLP
−0.3364
<0.0001
Grok
r3_inter , linear
−0.1537
0.0006
Grok
r3_inter , MLP
+0.0598
0.0010
Opus
Opus against Sonnet
s3_xor , linear
+0.2000
<0.0001
Opus
s3_xor , MLP
+0.3667
<0.0001
Opus
r4_sine , MLP
+0.9125
<0.0001
Opus
Table 3: Every paired comparison that survives the Holm correction within its family, with the first arm’s score minus the second’s on the same held-out rows. Classification is compared by an exact McNemar test on the discordant rows and regression by a paired bootstrap on the difference in squared error. Comparisons not listed were not separated after correction, including four that reach p<0.05 uncorrected and do not survive, of which the largest is the spiral rung discussed in the text.
Figure 8: Above, the four arms on every rung, each point the better of that arm’s two architectures, with the ceiling of the generating process drawn as a dashed line and the real rung carrying none. The arms are indistinguishable on the left of the figure, where the answer is a line and the ceiling is nearly one, and they fan out from the parity rung onwards. Sonnet is absent on three rungs because it returned no parameter vector there across four dispatches. Haiku is the only arm that falls below zero, which on a regression rung means a network worse than predicting the mean of the training targets, and it does so on two rungs while predicting beforehand that it would score above 0.55 on both. Below, what the hidden layer bought each arm against how far that rung’s training rows are from linearly separable, with the identity drawn for reference. The relationship reported for one arm earlier in this paper, that the gain tracks L2 where the construction succeeds, is a property of the stronger constructors rather than of the rungs: the points near the diagonal belong to the two arms that decline the hidden layer on exactly the five rungs where it is useless, and the points far below it belong to the arm that spends it where the optimum is linear.
arm
109 par.
ceilings
stop
tokens
modules
admissible
R2
qwen25_05b
0.49
8k/16k
length/length
16384
0/2
no
—
qwen25_15b
1.54
8k
stop
649
2/2
no
—
qwen25_3b
3.09
8k
stop
393
2/2
no
—
qwen3_4b
4.02
8k
length
8192
1/2
no
—
ministral8b
8.02
8k
stop
799
2/2
no
0.3245
Table 4: The open-weight size series on toy_line , the easiest rung of the suite, with the smallest model first. Every arm is four-bit quantised and served locally under the conditions fixed before the first dispatch, and no arm reached a second rung. “Ceilings” lists the reply budgets the arm was given in order, the second present only where the server ended the first for length; “stop” is the server’s own stop reason for each; “tokens” is the longest reply any dispatch produced. “Modules” counts the two architectures the rung asks for that arrived at all, “admissible” is the validator’s verdict on them unchanged, and the last column is the better architecture’s held-out coefficient of determination where one could be computed, against a floor of −0.0171 for the predictor that returns the training mean.