Local Gains and Fixed-Assignment Set Losses in Shared Set Decoders

Ze Zhang; Yang Zhang

cs.CVAug 12, 2026

Local Gains and Fixed-Assignment Set Losses in Shared Set Decoders

Authors: Ze Zhang, Yang Zhang

Organizations: School of Biomedical Engineering, ShanghaiTech University, Shanghai, China · Wuhan United Imaging Surgical Co., Ltd., Wuhan, China

Abstract

A query-relation deletion can improve the edited slot while reducing the utility of the prediction set that contains it. We study this tension in two related ResNet-50 DETR-family checkpoints using recorded, selection-conditional evidence from 710 paired image-relation units per checkpoint. The primary comparison subtracts a matched active control, which deletes the same leader source at a different recorded recipient, from the selected target deletion. It is therefore a composite contrast rather than a same-recipient placebo. The target-minus-control contrast is locally positive and fixed-assignment negative in both checkpoints. The opposite-sign pattern occurs within 302/710 DETR units and 460/710 DINO units. After rematching, the corresponding counts are 285/710 and 433/710. Rematching and native selection absorb enough of the mean loss for DETR intervals to cross zero, whereas DINO intervals remain negative, so persistence across readouts differs by checkpoint. A fixed-map comparison between hard deletion and a mass-preserving edit also differs before rematching. That comparison is conditional on the outcome-blind map and does not establish same-dose transport. Local intervention success therefore does not determine the consequence for a jointly decoded set. The supported conclusion is selection-conditional deletion sensitivity whose persistence depends on the readout and intervention operator. We do not identify an intervention-invariant edge mechanism, detector-level degradation, population prevalence, or the value of a training-time regularizer.

Figures & tables

Model	Endpoint	Estimate/count	Lower	Upper
DETR	Target–control local	$+0.095621$	$+0.070160$	$+0.121082$
DETR	Target–control fixed set	$-0.005741$	$-0.010889$	$-0.000592$
DETR	Target–control rematched	$-0.001575$	$-0.004544$	$+0.001393$
DETR	Target–control native	$-0.001686$	$-0.004741$	$+0.001370$
DETR	Local $+$ / target-set $-$	$302/710$	$38.95\%$	$46.20\%$
DETR	Local $+$ / rematched $-$	$285/710$	$36.60\%$	$43.79\%$

Table 1: Comparator-aligned primary results on the selected discovery population. Each checkpoint has 710 paired image–relation units. Target–control is a composite contrast because target and control edits use different recipients. Intervals are within-model simultaneous 95% intervals; event rows use Wilson intervals and the declared

10^{-4}

deadzone. All primary quantitative claims use the common estimand definitions specified in Section 3 .

Figure 1: A local query gain can coexist with a fixed-assignment set loss. A selected directed relation is hard-deleted in a jointly decoded query set and compared with a different-recipient active control. The target-slot response can be positive while fixed-assignment set utility is negative; fixed, rematched, and native readouts then show how much of that response persists. The lower panels separate the supported selection-conditional findings from mechanism, population-transport, and regularizer claims that the evidence does not identify.

Figure 2: Primary selected-deletion result. The target-minus-matched- active-control contrast is locally positive and fixed-assignment negative in both selected 710-unit populations, with frequent target-set and rematched within-unit sign reversals. DETR and DINO are shown separately. The active control is composite because target and control edits use different recipients; the rates are selection-conditional and are not arbitrary-relation prevalence.

Figure 3: Conditional mapped operator response. Panel (a) contrasts pre-softmax hard deletion with the post-softmax two-cell mass-preserving edit under the fixed outcome-blind map. Panels (b,c) show the conditional mapped contrast across local, derived, fixed-assignment, rematched, selection, and native readouts. The comparison is not same-dose and does not identify operator transport or a unique mechanism.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Arm	Recipient	Source/donor rule
Hard target	competitor	delete leader source before softmax
Hard active control	sham	delete same leader source at different recipient
Mass target	competitor	swap leader source with sham donor
Mass placebo	sham	swap leader source with competitor donor

Appendix

Table 2: Intervention-arm mapping at decoder layer 3. Arrows name recipients, sources, and donors; target and active-control arms use different recipients.

Figure 4: Supporting operator and context checks. The legacy mass-preserving conditions fail different components of the declared conjunction, while the fresh assay fails its required hard-deletion sensitivity check. These results do not establish population non-transport or regularizer performance.

Set Prediction

Set prediction models unordered collections of outputs or calibrated sets of plausible answers, with applications in vision, science, and decision-making.

See all 27 papers

From Spectral Saliency to Finite Deletion: Evaluating Selective Language Model Edits
Oct 13, 2026Ibne Farabi Shihab, Joyanta Jyoti MondalLanguage Model Editing LLM Pruning
Department of Computer and Information Sciences, University of Delaware, USA
Frozen DINO Localizes Image Edits Without a Localizer
Aug 19, 2026Zane Kumar, Vishal Jain, Bernhard KainzImage Forgery Localization Digital Image Forensics
Department of Computing St Paul’s School London London, United Kingdom · Department of Computing Imperial College London London, United Kingdom · Computing / AIBE Imperial College London / FAU London, UK / Erlangen, Germany
Support Operation Factorization: Compositional Readout of Frozen Vision Encoders under Controlled Interventions
Aug 6, 2026Zhongyao Wang, Wanli Ouyang, Taoyong Cui +1Visual Representation Learning Representation Disentanglement
Fudan University · MMLab, The Chinese University of Hong Kong · Department of Computer Science and Engineering, The Chinese University of Hong Kong

Oct 13, 2026J/K move · Enter open · S save

From Spectral Saliency to Finite Deletion: Evaluating Selective Language Model Edits

Ibne Farabi Shihab, Joyanta Jyoti Mondal

Department of Computer and Information Sciences, University of Delaware, USA

Language Model Editing LLM Pruning

Open paper AskPDF arXiv

Deleting singular components offers a precise way to remove a capability from a language model, yet the first-order spectral saliency used to rank components predicts only a local loss change, whereas deletion is a finite intervention. In this paper, we study how prediction error affects selective editing, where selective editing requires degradation of a target task while preserving others. We decompose the error into derivative transfer across item splits and nonlinear response along the deletion path. We establish that outer interval screening retains every feasible candidate when every candidate's intervals contain its finite validation loss changes on every task. We evaluate these effects using one answer-only negative log-likelihood objective for search, validation, and test, with disjoint calibration and evaluation masks. The primary configuration targets arithmetic in Mistral-7B-Instruct-v0.3 while preserving code and question answering; extensions cover Llama-3.1-8B-Instruct and reversed target roles. In the primary study, which deletes 1024 coordinates from a support of 7168, calibrated intervals cover 968 of 1024 evaluation masks. Outer screening reaches the direct-selected edit after 25 verification checks, whereas direct verification abstains at a 32-check cap, but it rejects 4 of 74 feasible masks. All three selected edits pass validation, yet only one meets every test constraint. Interval coverage, feasible-candidate retention, and independent test preservation are therefore distinct properties and require separate assessment.

Abstract

Set Prediction

Explore similar work

From Spectral Saliency to Finite Deletion: Evaluating Selective Language Model Edits

Frozen DINO Localizes Image Edits Without a Localizer

Support Operation Factorization: Compositional Readout of Frozen Vision Encoders under Controlled Interventions