Representable but Unlearned: Encoding Rank and the Interaction-Prediction Floor

Zahra Khodagholi; Niloofar Yousefi

cs.LGSep 28, 2026

Representable but Unlearned: Encoding Rank and the Interaction-Prediction Floor

Authors: Zahra Khodagholi, Niloofar Yousefi

Organizations: University of Central Florida Orlando, Florida, USA

Abstract

Input encodings can restrict which measured contrasts a predictor can jointly reproduce, even when no single contrast is forced to vanish. We compute the attainable contrast space from an encoder's equivalence classes and a fixed contrast design, without labels, loss, or a fitted model; projecting the recorded contrasts onto that space gives an empirical error floor for any unrestricted decoder on those classes. On a 140-rectangle siRNA interaction panel, a graph neural network's training-only feature mask merges 165 endpoint states into 90 classes and cuts the rank of the 140 interaction contrasts to 72. The resulting floor is 0.009980, which is 14.6% of the fitted model's interaction squared error; the fitted model reaches 0.068335, slightly worse than a control predicting no interaction at all. A minimum of three restored chemistry columns recovers full rank. Refitting without the mask removes the floor entirely, yet interaction MSE improves by only 0.000017 under the reported protocol, and the restored columns remain absent from every training input. On a released RNA-splicing predictor, whose encoding is injective on the measured states, the same computation returns the full design rank of 1,986 and a floor of exactly zero. These results separate what an encoding permits from what a fitted model achieves; they do not identify what limits the remaining error. The rank check needs no fits and bounds what any amount of training under a fixed encoding can recover. The project repository is available at https://github.com/shadi97kh/REPRESENTABLE-BUT-UNLEARNED.

Figures & tables

Figure 1: Architecture and evaluation (Appendix A2 ). Panels (a)–(d) show the siRNA representation, typed message passing, ordered readout, and evaluation with frozen weights. The two supervision bands distinguish activity training from pair-supervised follow-up; held-out labels enter evaluation only. Band (e) summarizes saved-model auditing, published-predictor evaluation and controlled refits: 56 siRNA mask-restoration fits and 12 MAVE-NN attempts, six of which supply reported results.

Family	Method	$q$	Floor	In-span	$\mathrm{MSE}_{q}$	Ratio
Antisense	R1 GNN	14	0.018972	0.030885	0.049856	2.627909
Antisense	R1 no-message	14	0.018972	0.031574	0.050546	2.664273
Antisense	Chemistry tree	14	0.018972	0.023931	0.042903	2.261392
Antisense	Token CNN	14	0.018972	0.030339	0.049311	2.599177
Sense	R1 GNN	10	$1.478\times 10^{-5}$	0.003320	0.003335	225.665139
Sense	R1 no-message	10	$1.478\times 10^{-5}$	0.003221	0.003236	218.972986

Table 1: Encoding-floor decomposition of the recorded B3 error, equal weights: Floor is

q^{-1}\lVert(\mathrm{Id}-P)\widetilde{I}\rVert^{2}

with

P

the projector onto

\operatorname{col}(CH)

for that family, In-span is

q^{-1}\lVert P\widetilde{I}-p\rVert^{2}

, and

\mathrm{MSE}_{q}

their sum, exact before rounding. The floor is shared across methods in a family because every fitted contrast vector lies in

\operatorname{col}(CH)

; denominators are the 14 and 10 marginal contrasts and the 140 rectangles. All entries are rounded from stored values; floor, in-span and ratio are each computed before rounding. The tree is extremely randomized trees, the token CNN a convolutional baseline.

Grouped benchmark: 1,924 of 2,927 rows (65.73%) from one source in one sequence component
Sensitivity	Method	Grouped MSE	$R^{2}$	APP MSE	$R^{2}$	S7 MSE	$R^{2}$
Source-excluded	R1 GNN	0.0641	-0.212	0.1631	-0.005	0.1191	-0.324
	R1 no-message	0.0625	-0.182	0.1637	-0.009	0.1221	-0.358
	Chemistry tree	0.0529	-0.001	0.1655	-0.020	0.0974	-0.082
	Token CNN	0.0628	-0.187	0.1626	-0.002	0.1103	-0.226
	Training row mean	0.0571	-0.079	0.1810	-0.115	0.1587	-0.764

Table 2: Row-weighted fixed-ensemble prediction on matched rows within each sensitivity; cohorts and models are those of Section 1.1 and Table 1 . Grouped coverage is 1,003 and 2,607 rows; APP and S7 retain 1,839 and 118. The training row mean is a deployable constant from each training partition; the oracle test mean reads the evaluation labels and is a diagnostic, not a risk floor.

Panel/model	Inputs	Forced	$\mathrm{MSE}_{165}$	$(\Delta\mu)^{2}$	AS	SS	NA	Pred. NA
Measured panel	165	–	0.079894	–	0.041341	0.020415	0.018138	–
R1 GNN	90	0	0.077350	0.000041	0.042930	0.016196	0.018183	0.000002
R1 no-message	90	0	0.077353	0.000005	0.042811	0.016371	0.018166	0.000002
Chemistry tree	90	0	0.078546	0.000012	0.043859	0.016079	0.018596	0.000146
Token CNN	90	0	0.077276	0.000211	0.042819	0.016072	0.018174	0.000005

Table 3: Primary B3 on the complete 15 by 11 panel, equal weights, reusing source-held-out endpoints. Exact complete inputs among the 165 states force no cancellation in any of the 140 rectangles, for all ten preprocessing instances.

\mathrm{MSE}_{165}

is endpoint MSE over 165 states, not the contrast

\mathrm{MSE}_{q}

of Table 1 ; AS, SS and NA are antisense, sense and non-additive components, and row one, being centered energy, has no grand-mean term. The four components sum to

\mathrm{MSE}_{165}

exactly; Pred. NA is each model's own predicted nonadditive energy. These component sums hold to

10^{-15}

Quantity	$N$	Model MSE	Matched control	Difference	Pearson
Endpoints, released test split	6078	0.084224	0.259250	-0.175026	0.822247
Endpoints, rectangle population	3696	0.089748	0.246508	-0.156759	0.797282
Single substitutions	12486	0.155262	0.302181	-0.146919	0.697187
Interactions (primary)	2083	0.361690	0.456701	-0.095011	0.462959
Interactions, library 2	1947	0.327946	0.455823	-0.127877	0.531564

Table 4: Released MAVE-NN pairwise GE predictor on the released MPSA test split, in

\log_{10}

PSI; the checkpoint-provenance limitation is stated in Appendix A10.21 . Endpoint controls are the constant fitted to the released training-plus-validation population; contrast controls are matched zero-effect predictions on identical identities and weights. Primary rectangle weights give equal mass to each position pair, then each background within a pair, then each rectangle within a background; for library 2 they are restricted to surviving identities and renormalized. Negative differences favour the model; differences are computed before rounding.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Meaning
$y_{i}$	Inhibition fraction, by cohort: the grouped benchmark uses the released inhibition fraction; APP uses $(100-\text{primary mean percent mRNA remaining})/100$ ; the primary B3 panel uses $1-\text{relative eGFP}$ (Appendix A5.1 ); Davis S7 is kept on its original response scale. No clipping.
$\mu(s,m,a)$	Population mean of that assay endpoint; not directly known from a reported sample mean.
$\hat{f}_{k}(x_{i})$	Committed prediction of model $k$ from the permitted molecule/context features $x_{i}$ .
$g(i),q(i)$	Study/source-linked component and sequence-only component, respectively.
$w_{i}$	Within-partition observation weight $n/(Gn_{g(i)})$ , where $G$ is the number of study components and $n_{g}$ is its size.
$X,A,b,C$	Node features, relation adjacency tensor, node mask, and training-masked numeric context; here $C$ is the context tensor, not the contrast matrix $C$ of Section 2 .

Appendix

Table 5: Notation and conventions.

Primary subscript	Frozen name
OMe	2-O-Methyl
F	2-Fluoro
DNA	2-Deoxy
AEM	2-Aminoethoxymethyl
APM	2-Aminopropoxymethyl
EA	2-Aminoethyl

Appendix

Table 6: Literal source aliases used in primary B3 admission. AP was excluded from the original B3 admission; its primary sugar alias and duplicate release mappings are adjudicated in Appendix A8 .

Reconciliation diagnostic	Count/value
identity_matched_pairs	1604
within_half_tenth_percentage_point	1209
within_half_percentage_point	1258
greater_than_five_percentage_points	299
max_absolute_activity_difference	1.0852170324990895

Appendix

Table 7: Primary/release audit conditional on the stated identity mapping. Rounding tolerances are activity fractions; no frozen label is changed.

Cohort	Model	MSE	$R^{2}$	$r$	Pred. SD	Bias
APP	R1 GNN	0.1625	-0.001	-0.018	0.0072	-0.0081
	R1 no-message	0.1630	-0.005	0.015	0.0085	-0.0286
	R0 GNN	0.1625	-0.001	-0.026	0.0073	-0.0049
	R0 no-message	0.1629	-0.004	-0.001	0.0085	-0.0241
	Chemistry tree	0.1896	-0.168	-0.021	0.0103	-0.1644
	Token CNN	0.1644	-0.013	0.080	0.0036	-0.0483

Appendix

Table 8: All original v3 ensemble activity procedures with row weighting on the full cohorts: APP 1,839 rows, S7 118 and the grouped benchmark 2,927, with label SD 0.4028, 0.3000 and 0.2682. R1 is the training-gated and R0 the original GNN. Training means are constants fitted on each training partition; the grouped cohort pools five outer test folds, so they vary there and have a defined correlation. The oracle test mean reads the evaluation labels and is a diagnostic, not a risk floor. A dash marks a correlation undefined for a constant prediction; bias is mean prediction minus mean label. Alternate weightings and further metrics: v3/tables/activity_metrics.csv .

Quantity and scope	Actual value
Core fit objects	2312
Executed optimizer updates	1407967
Selected-checkpoint updates	406367
Summed core-fit CPU seconds	12523.769
GPU training-region elapsed seconds	11689.879
Completed stage child CPU seconds	13202.199

Appendix

Table 9: Resource accounting with distinct scopes. The core-fit and process-cost entries summarize the historical campaign; the intervention rows separately report new work. These entries do not form one cumulative resource total. Fit-region, stage and GPU-region time measurements overlap and are not added. The MAVE-NN attempt count includes both invocations; the retained-fit update count and incomplete cumulative accounting are described in Appendix A10.24 .

Sensitivity	Method	$m/q$	Model MSE	Zero MSE	$10^{3}\Delta$	Flips
Omit AS	Tree	154/130	0.059628–0.077251	0.056062–0.073257	2.456–4.031	0
Omit AS	GNN	154/130	0.056195–0.073399	0.056062–0.073257	0.015–0.143	0
Omit AS	No-msg	154/130	0.056160–0.073339	0.056062–0.073257	-0.045–0.098	1
Omit AS	CNN	154/130	0.056290–0.073475	0.056062–0.073257	0.007–0.229	0
Omit SS	Tree	150/126	0.055818–0.079524	0.053743–0.075358	1.176–4.188	0
Omit SS	GNN	150/126	0.053882–0.075507	0.053743–0.075358	-0.009–0.151	1

Appendix

Table 10: Ranges over three reference shifts, fourteen antisense omissions or ten sense omissions.

m/q

: retained endpoints/contrasts. Paired differences

\Delta

are scaled by

10^{3}

and computed before taking ranges, not by subtracting range endpoints. Flips count reversals from the unperturbed ordering. These are dependent descriptive diagnostics, not confidence limits.

Sensitivity	Method	Predicted SD	Recorded SD	Largest influence	Gap change
Omit AS	Tree	0.008806–0.018376	0.204264–0.229500	JC10	-0.001278
Omit AS	GNN	0.000252–0.001648	0.204264–0.229500	JC10	-0.000115
Omit AS	No-msg	0.000299–0.001714	0.204264–0.229500	JC10	-0.000122
Omit AS	CNN	0.001113–0.002984	0.204264–0.229500	JC10	-0.000197
Omit SS	Tree	0.007219–0.018653	0.203193–0.230994	JC5	-0.002558
Omit SS	GNN	0.000327–0.001674	0.203193–0.230994	JC5	-0.000139

Appendix

Table 11: Spread ranges and the omitted margin (or reference multiplier) with largest absolute change of paired error difference. A positive reference multiplier adds one reported SD to the shared activity reference.

Archive path	Preserved evidence
v4/adjudication/row_reconciliation.csv	All 1,972 raw source rows (1,924 admitted): cells, chemistry, cardinality, original/primary values, categories
v4/dataset/primary_reanchored_observations.jsonl	2,607 versioned observations, original values and graph indices
v4/review_checks/source_scores.csv	All-source, Bramsen, other-source, every individual source and constant; three weightings
v4/evaluation/per_source_metrics.csv	Every source after focused refitting; exact matched coverage
v4/evaluation/ensemble_predictions.csv	Exact member means and cohort identifiers
v4/fits/	Best/last checkpoints, optimizer states, histories, selected transforms and memberships

Appendix

Table 12: Indexed detailed evidence; no repetitive per-fit dump is printed in the manuscript.

Cohort	Original v3 model	MSE	$R^{2}$	$r$	Pred. SD	Bias
APP	R1 GNN	0.1625	-0.001	-0.018	0.0072	-0.0081
	R1 no-message	0.1630	-0.005	0.015	0.0085	-0.0286
	Chemistry tree	0.1896	-0.168	-0.021	0.0103	-0.1644
	Token CNN	0.1644	-0.013	0.080	0.0036	-0.0483
S7	R1 GNN	0.1093	-0.215	0.112	0.0059	0.1403
	R1 no-message	0.1047	-0.164	0.117	0.0066	0.1231

Appendix

Table 13: Unchanged v3 released-trained activity results on original response scales. Label SD is 0.4028 on APP and 0.3000 on S7. Prediction spread and mean calibration are distinct from discrimination; no test-fitted adjustment is used.

Method	MSE	$R^{2}$	$r$	Pred. SD	Label SD	Retro. mean	Train const
GNN	0.077350	0.031844	0.184788	0.040265	0.282656	0.079894	0.080173
No-msg	0.077353	0.031810	0.185135	0.038465	0.282656	0.079894	0.080173
Tree	0.078546	0.016873	0.167266	0.076857	0.282656	0.079894	0.080173
CNN	0.077276	0.032767	0.190711	0.045136	0.282656	0.079894	0.080173

Appendix

Table 14: Endpoint prediction over the 165 deduplicated states on the original response scale. The retrospective column is the MSE of the evaluation-mean predictor; the final column is the MSE of a genuine training-fitted constant recovered from the ten fit records, never from these evaluation labels.

Method	Family	$n$	MSE	Zero MSE	$r$	Pred. SD	Label SD
GNN	AS effect	14	0.049856	0.051986	0.028779	0.024352	0.170616
GNN	SS effect	10	0.003335	0.000855	0.286065	0.034469	0.013953
GNN	Interaction	140	0.068335	0.068205	-0.090263	0.001589	0.222442
No-msg	AS effect	14	0.050546	0.051986	0.019753	0.021060	0.170616
No-msg	SS effect	10	0.003236	0.000855	0.269464	0.034474	0.013953
No-msg	Interaction	140	0.068281	0.068205	-0.034429	0.001652	0.222442

Appendix

Table 15: The three effect families with directions and response scale fixed: antisense

f(A,0)-f(0,0)

, sense

f(0,B)-f(0,0)

and the original reference-based interaction. Seed dispersion is separate and no independent-marginal interval is claimed.

Strand	Indistinguishable variants	Positions	Size
AS	JC-A1, JC-F1, JC-S1	3, 18	3
AS	JC-A2, JC-F2, JC-S2	4, 18	3
AS	JC-A3, JC-F3, JC-S3	3, 4, 18	3
SS	DO003, DO004	17	2

Appendix

Table 16: All non-singleton strand classes, determined by exact complete inputs across every partner. Other strand states are singletons. Full positional names, raw records and equality witnesses remain in the store.

Scen.	Weights	Model	Constant	Difference	Lower	Upper	Grp
primary	rows	GNN	row const	-0.001217	-0.004579	0.020462	10
primary	rows	GNN	grp const	-0.006226	-0.010077	0.021086	10
primary	rows	Tree	row const	-0.006784	-0.013080	0.003109	10
primary	rows	Tree	grp const	-0.011792	-0.016886	0.000805	10
primary	equal study	GNN	row const	0.015532	-0.007321	0.038432	10
primary	equal study	GNN	grp const	0.015232	-0.007620	0.038511	10

Appendix

Table 17: Fixed-ensemble error minus each training-fitted constant, with paired conditional 95% percentile intervals from resampling whole study-linked components under both weightings at the existing fixed analysis seed over 10,000 draws, predictions and selection held fixed. Constants are training-fitted and never an evaluation mean. The tree has lower observed MSE than both constants under row weighting, but not against the group-mean predictor under equal-component weighting on the excluded cohort; all four source-excluded tree-versus-constant intervals include zero. Ten and nine components respectively: conditional, few-component intervals whose endpoints are not subtracted.

Method	Seed	Scale	Mean ovl.	Min ovl.
GNN	1103	-0.250000	1.000000	1.000000
GNN	1103	0.250000	1.000000	1.000000
GNN	2207	-0.250000	1.000000	1.000000
GNN	2207	0.250000	1.000000	1.000000
GNN	3301	-0.250000	1.000000	1.000000
GNN	3301	0.250000	1.000000	1.000000

Appendix

Table 18: Within each of the seventeen exact APP assay pools, top-five selection mass under the existing fractional boundary-tie rule, and average-rank displacement normalised by

n-1

. The pools overlap and come from one patent family, so these are not independent trials.

Arm	Seed	Scale
R1	1103	-0.250000
R1	1103	0.250000
R1	2207	-0.250000
R1	2207	0.250000
R1	3301	-0.250000
R1	3301	0.250000

Appendix

Table 19: Raw input-gradient changes on the same sixteen fixed APP queries. Each statistic maximizes across those queries; the coordinate statistic also maximizes across coordinates. Relative change divides each query gradient-change norm by its baseline norm, with threshold

10^{-8}

; no near-zero references occur. Other graph inputs are fixed. These are local encoded sensitivities, not feasible chemical interventions, and R1 is the expected structural null control.

Protocol	GNN	No-message	Tree	CNN
v1 original	0.521	0.497	0.565	0.453
v2 factorial	0.538	0.498	0.574	0.530
v2 deployment	0.397	0.367	0.631	0.510
v3 group selected	0.534	0.577	0.445	0.583
Source-excluded	0.400	0.374	0.696	0.421
Primary-assay	0.565	0.588	0.514	0.563

Appendix

Table 20: Common tie-aware APP mean top-five percentile across the identical seventeen assay pools; larger is preferred. The pools overlap and come from one patent family, so they are not seventeen independent experiments.

Method	Row MSE	Comp. MSE	$r$	Pred. SD	Sign
Activity GNN	0.2351	0.3560	0.083	0.036	111/137
Pair GNN	0.2221	0.3211	0.248	0.038	111/137
Pair no-message	0.2220	0.3196	0.228	0.043	111/137
Pair tree	0.2438	0.3101	0.120	0.167	109/137
Pair ridge	0.2193	0.2888	0.226	0.148	105/137
Zero	0.2808	0.4342	–	0.000	0/137

Appendix

Table 21: Unchanged held-out measured B2 errors. Comp. gives each of twelve sequence components equal mass. Sign counts use 137 non-ties under the operational 0.02 band; the majority control also reaches 111/137.

Retention score	Model MSE	Zero MSE	Excess MSE
Support absence	0.434741	0.434166	+0.000575
Probe sensitivity	0.356843	0.356330	+0.000513
Ensemble SD	0.363586	0.362827	+0.000759
Chemical novelty	0.434741	0.434166	+0.000575
Random	0.451729	0.451135	+0.000594

Appendix

Table 22: R0 B2 at 60% observation mass: 93.6 fractional pairs out of 156, twelve sequence components. Each zero control uses identical retained membership and weights. Random averages 200 frozen selections. These three-seed activity-only predictions are distinct from historical pair-supervised estimates.

Coverage	Pair mass	Probe MSE	Its zero MSE	SD MSE	$\Delta E$	95% interval
20%	31.2	0.148697	0.150237	0.408344	-0.002887	[-0.004963,-0.000467]
40%	62.4	0.273094	0.273292	0.425092	-0.001938	[-0.003812,-0.000147]
60%	93.6	0.356843	0.356330	0.363586	-0.000246	[-0.001059,+0.000561]
80%	124.8	0.336510	0.336282	0.322389	-0.000208	[-0.000668,+0.000103]
100%	156.0	0.434741	0.434166	0.434741	+0.000000	[+0.000000,+0.000000]

Appendix

Table 23: R0 B2 across all frozen coverage levels. Delta E is probe minus disagreement in excess-over-identical-subset-zero risk; 60 percent is primary. Equal-component weights are renormalized after fractional retention. Different methods retain different zero risks.

Panel	$m$	$k$	$q$	$\operatorname{rank}C$	$\operatorname{rank}CH$	$d_{\mathrm{enc}}$	Floor	Energy	Share
B3 (reproduced)	165	90	140	140	72	68	0.009980	0.068205	14.63%
Bramsen, expanded	1932	1183	1845	1845	1115	730	0.007565	0.055180	13.71%
new contrasts only	1792	1111	1705	1705	1043	662	0.007367	0.054111	13.61%
MAVE-NN splicing	3696	3696	2083	1986	1986	0	0	0.412839	0%
GB1 binding	17689	17689	15856	15856	15856	0	0	0.549741	0%

Appendix

Table 24: Encoding rank and empirical floor on further measured panels, equal weights

1/q

d_{\mathrm{enc}}

counts restrictions added by the encoding;

q-\operatorname{rank}C

counts design dependencies (97 for splicing, none elsewhere). Energy is

q^{-1}\lVert\widetilde{I}\rVert^{2}

and Share is the floor as a percentage of it, a different denominator from the fitted-error shares of Table 1 . Zero floors are exact:

H

is injective, so

\operatorname{rank}(CH)=\operatorname{rank}(C)

and the recorded contrasts lie in

\operatorname{col}(CH)

. The new-contrasts row is a follow-up check run after the frozen audit. Response scales differ between panels, so floors are compared within a panel, not across panels.

Explore similar work

Aug 12, 2026cs.LG

Low-Interaction-Rank Learning: Unifying Multiplicative Dual-Encoder Heads

A multiplicative dual-encoder network computes a real-valued output for a pair of inputs as the inner product of their separate encodings. This architecture has been developed independently in operator learning, bipartite matching, contrastive vision-language models, retrieval, and other areas, yet no unified theory guides the basic design decisions: how many interaction modes to represent, how to normalize the encoders, and when the architecture should be avoided. We provide such a foundation by introducing the class of functions of low interaction rank, a class whose intrinsic complexity is measured by its interaction spectrum. Within this framework, approximation error decomposes into a spectral truncation term and an encoder-realization term; sample complexity is governed by the sum of the two encoder complexities rather than their product; and a usability criterion based on spectral decay determines when the architecture can succeed. The same framework exposes a central identifiability problem: the encoders are defined only up to a linear gauge symmetry that leaves the learned coordinates arbitrary. We show that normalization is gauge fixing and that whitening pins the interaction modes up to permutation and sign, thereby explaining the uninterpretability of contrastive dimensions and providing a constructive remedy. Experiments on synthetic kernels, operator learning, and CLIP models validate the theoretical predictions: spectral decay rates match the predicted scaling, whitening recovers the true modes, and independently trained CLIP models are related by a single rotation which, after removal by whitening, exposes interpretable concept axes. The code of this paper is provided at https://github.com/RS2002/Mul-Net .

Zijian Zhao, Sen Li

The Hong Kong University of Science and Technology · The Hong Kong University of Science and Technology (Guangzhou)

Dual-Encoder Architectures Low-Rank Structure Unimodal Encoders

Ask PDF

Sep 2, 2026cs.LG

The Gradient Does Not See Rank: Rank-Indifference in Matrix-CODI on ProsQA

Continuous chain-of-thought models compress reasoning into latent tokens. Matrix-valued variants, which route each latent token through a d x d matrix bottleneck, introduce rank as a single-sample structural observable on the latent matrix Z. If matrix latents carry parallel reasoning paths via superposition, rank should track them, and truncating Z to low rank should hurt accuracy on tasks whose solutions plausibly require multiple components. Across four training regimes of a matrix-CODI model (three on ProsQA, one on GSM8K-Aug below the learning threshold), the rank-k projection ablation curve is flat to within 0.6 percentage points. A three-seed replication yields 81.0 +/- 2.0 percentage points accuracy while the final effective rank of Z spans {4, 12, 13}; the loss does not reward any particular rank. To test whether rank-blindness arises from the flatten-then-project readout alone, we trained four readouts: a bilinear reparametrization, a bilinear-plus-GELU readout nonlinear in Z, an SVD-augmented readout feeding singular values through an MLP, and a quadratic readout in Z Z^T. All four rank-k curves remain flat (Spearman p-values 0.63, 0.14, 0.82, 0.46). The flat curves persist for readouts nonlinear in Z. A linear probe on Z underperforms a raw pretrained hidden state at target prediction (AUC 0.673 vs. 0.846). A negative control on vanilla GPT-2 SFT (no matrix bottleneck, no Z, three seeds, n=500) reproduces a flat rank-k curve under the same intervention paradigm with pooled-mean range 0.20pp, and a random-h sensitivity floor lands at the same accuracy: the rank-k ablation alone conflates rank-blindness with position-irrelevance.

Samuel Larson

Pebble ML

Low-Rank Structure Matrix Ranking Ablation Readout Gradient+1

Ask PDF

Sep 30, 2026cs.LG

One pool, many targets: a conservation layer and what archival data can identify

Pairwise guide--transcript scores do not enforce conservation of a finite guide-loaded RISC pool when they are interpreted independently as occupancies. We formulate a differentiable scalar equilibrium layer: one conservation equation with a unique positive root and exact implicit gradients. It yields a redistribution theorem, a qualified high-resource limit, an analysis of the retrieval approximation, and a conditional rank-invariance result: within one construct at one dose, rankings by fractional occupancy cannot distinguish equilibrium from independent scoring. We therefore audit the two experiments that proposition leaves open, dose and cross-context, on archival off-target data. Corrected thermodynamic affinities associate weakly with measured repression in the direction a working predictor requires, but a paired permutation test and a construct-cluster bootstrap do not establish added predictive value from the coupling: what survives their differing permutation-null baselines is \GapNet{}, a descriptive \GapNetOverSE{} of the equilibrium association's cluster standard error. The dose fits are heterogeneous and frequently violate the model-implied exponent constraint, which is superlinear rather than sublinear, so these data do not identify the competition parameter. A saturable compression of the competitor set holds both accuracy targets on held-out guide families but is not faster at the size measured. The contribution is a reusable conservation operator and the experimental information needed to test it. The code for this study is available at https://github.com/shadi97kh/One-Pool-Many-Targets.

Zahra Khodagholi, Niloofar Yousefi

Department of Industrial Engineering University of Central Florida

Feature Alignment Distributional Fidelity Equilibrium Exponents Archive Risc-V+1

Ask PDF