Human corrections identify editable spans, but the examples receiving corrections may come from a selective feedback channel. We analyze this interaction at a fixed model checkpoint by decomposing a localized gradient into edited and retained untouched components. Squared relative selection bias is a ratio of quadratics whose derivative has the sign of an explicit quadratic polynomial. Localization can increase, decrease, or nonmonotonically change this diagnostic; its direction depends on component biases and geometry. Oracle importance weighting recovers the population mean for each fixed localization objective, but these objectives have different targets. Against one common full-gradient target, we derive the finite-sample mean-squared error, an analytic optimal retention coefficient, and a fixed-clipping extension. Exact finite-population calculations and 10,000 Monte Carlo repetitions per sample size verify the identities and counterexamples. Public human-post-edit experiments use two translation directions and pretrained models, with declared synthetic selection. An English-German extension differentiates 73.89 million native parameters. In all three declared settings, hard localization has higher relative bias but lower absolute bias than full retention. Untouched-component biases are nonzero and selected component means have negative inner products, so the general criterion applies where the simple unbiased/aligned explanation fails. Output-bias diagnostics show the same endpoint ordering of relative bias across both directions, with one interior maximum. Mechanisms reuse each language's records and include a mixture; they are not independent replications. The evidence separates relative amplification from absolute gradient error and establishes estimation properties, without inferring translation-quality gains or identifying actual complaint propensities.
Figures & tables
Figure 1: Exact finite-population geometry. Smaller α means stronger localization. A: unbiased untouched gradients and aligned means yield relative-bias amplification as α decreases, although absolute bias stays fixed. B: selection in the untouched component reverses the effect. C: opposing means produce an interior maximum. Values are exact population quantities; no sampling intervals are needed.
Figure 2: Exact MSE to one full traffic-gradient target. Left: the oracle IPW optimum moves toward full retention as sample size increases; dots show the analytic minimizers. Right: joint retention and population-normalized clipping frontier at n=4 . The cap grid contains 100 log-spaced values, and retention is optimized analytically at each cap. The marked solution is the best on that cap grid, not a claim of a universal deployment setting.
n
Estimator
MSE to μ1
Monte Carlo SE
16
Unweighted
0.7842
0.0050
16
Oracle IPW
0.5487
0.0082
16
Self-normalized
0.5480
0.0050
16
Population-clipped ( M=2 )
0.3987
0.0039
256
Unweighted
0.6375
0.0012
256
Oracle IPW
0.0339
0.0004
Table 1: Executed Monte Carlo results for full retention ( α=1 ), using 10,000 repetitions per sample size. The clipped estimator uses a fixed population normalizer and cap M=2 , while the self-normalized estimator has a random sample normalizer. Lower MSE is better for this declared gradient target.
Figure 3: Finite-sample error to the same full-gradient target. Left: hard localization remains mismatched even as oracle IPW estimates its own localized target accurately. Right: full-retention oracle IPW and self-normalization approach the full target, while fixed clipping retains asymptotic bias. Error bars show approximate 95% Monte Carlo intervals for mean MSE.
Direction
Selection feature
ρ(0)
ρ(1)
∥ε∥
en-de
Edit fraction
0.454
0.347
0.0031
en-de
Source length
0.210
0.131
0.0014
en-de
Both
0.227
0.154
0.0016
en-zh
Edit fraction
0.220
0.088
0.0044
en-zh
Source length
0.093
0.037
0.0021
en-zh
Both
0.115
0.053
0.0022
Table 2: Shared output-bias gradients under synthetic selection, with 256 reporting records per direction. The ratio uses the selected mean norm as in Equation ( 4 ); ε=bU is the untouched-component bias. Values are exact for the declared finite populations, up to numerical precision.
Figure 4: Measured output-bias geometry on public human post-edits, under synthetic selection. Lines evaluate the exact ratio of quadratics from the stored population means; grid points are separately enumerated. English–German edit-based selection has a small interior maximum at α≈0.026984 , retained in the analysis despite the larger endpoint difference.
Selection
ρQ(0)
ρQ(1)
∥b0∥
∥b1∥
∥ε∥
Edit
0.44748
0.33102
0.49340
0.65602
0.47136
Length
0.19267
0.10497
0.12684
0.20144
0.16385
Edit + length
0.22455
0.14286
0.18330
0.27333
0.21668
Table 3: Completed native-parameter gradient diagnostics on the fixed public reporting population. The primary relative measure uses the selected mean in its denominator. Hard localization has larger relative bias but smaller absolute bias than full retention in all three settings. These are finite-population gradient measurements under declared synthetic selection, not translation-quality changes.
Figure 5: Native-parameter relative and absolute gradient bias. Curves are evaluated from the saved six moments; 21 grid points and the exact derivative coefficients are exported. The two panels expose the distinction between a normalized discrepancy and its numerator. No parameter update is performed.
Deleting singular components offers a precise way to remove a capability from a language model, yet the first-order spectral saliency used to rank components predicts only a local loss change, whereas deletion is a finite intervention. In this paper, we study how prediction error affects selective editing, where selective editing requires degradation of a target task while preserving others. We decompose the error into derivative transfer across item splits and nonlinear response along the deletion path. We establish that outer interval screening retains every feasible candidate when every candidate's intervals contain its finite validation loss changes on every task. We evaluate these effects using one answer-only negative log-likelihood objective for search, validation, and test, with disjoint calibration and evaluation masks. The primary configuration targets arithmetic in Mistral-7B-Instruct-v0.3 while preserving code and question answering; extensions cover Llama-3.1-8B-Instruct and reversed target roles. In the primary study, which deletes 1024 coordinates from a support of 7168, calibrated intervals cover 968 of 1024 evaluation masks. Outer screening reaches the direct-selected edit after 25 verification checks, whereas direct verification abstains at a 32-check cap, but it rejects 4 of 74 feasible masks. All three selected edits pass validation, yet only one meets every test constraint. Interval coverage, feasible-candidate retention, and independent test preservation are therefore distinct properties and require separate assessment.
Ibne Farabi Shihab, Joyanta Jyoti Mondal
Department of Computer and Information Sciences, University of Delaware, USA
We derive exact local responses for attention interventions, allowing candidate edits to be scored from a cached baseline and one backward pass. The starting point is the RoPE derivative ∂pz(p)=Az(p): its integral gives the finite positional displacement, which we carry through the softmax without linearising either rotation or normalisation. The resulting predictions achieve 95.36--96.52% sign accuracy across 92,160 executed positional edits on 768 held-out prompt sets, reducing answer-margin MAE by 73.6--82.5% against the positional Jacobian and by 36.2--50.9% against zero. For simultaneous key and value edits, the same divided-difference calculus isolates the interaction term CKV=∑j(pj′−pj)εj, which is omitted by adding separate attributions. Retaining it reduces downstream margin MAE by more than a factor of nine in every setting of a 5,120-intervention sweep across two Qwen sizes, two tasks, and multiple layers; reductions against a quadratic interaction correction are 75.9--98.5%. Exactness concerns the edited attention write; downstream predictions contract that response with a baseline gradient and are evaluated by native execution. The calculus also yields a KL certificate for local approximation error, an exact query-conditioned gradient-step representation whose curvature identifies attention-preserving query directions, and minimum-norm query control. Sparse evaluation supports candidate ranking and cache decisions under explicit local distortion criteria.
Julie Huang, Maggie Chlon, Gregory Gutin +1
Hassana Labs · Department of Computer Science, Royal Holloway, University of London · University of Oxford
Data attribution methods using gradient similarity are widely used to analyze and select training data for large language models, but what gradient similarity actually measures is debated. Some interpret it as identifying task-relevant skills, while other work reports that surface form is the main factor. We resolve this debate for supervised fine-tuning examples by varying task and answer format independently. Specifically, we render benchmarks in different answer formats, such that datasets can share a task without a format or a format without a task. We find that gradient alignment follows the answer format, as benchmark pairs sharing an answer format align strongly (disattenuated cosine near 0.4), while same benchmarks rendered with different answer format classes show no alignment (near 0.0). We demonstrate that this ordering holds from the earliest pretraining checkpoints through post-training, and across model scales and families. We then analyze the released selections of LESS, a gradient-based data selection method for instruction tuning, and find that each target's selections over-represent the target's own answer format. Hence, we demonstrate that gradient-based attribution methods track format similarity more than task semantics, meaning that such methods, as well as the semantic interpretation of the gradient, should be tested on data where answer format and task vary independently for greater robustness and reliability.