Adapters are often selected and deployed based on nominal labels (e.g., instruction-tuned), which implicitly suggest what capability improves after adaptation. We test whether nominal training objectives reliably align with realized cross-task capability gains by evaluating the same LoRA adapter across tasks. Our strongest evidence is tied to strict, automatically verifiable instruction following as measured by IFEval: across multiple seeds, base models, and LoRA settings, nominal labels recurrently but not universally fail to predict improvements on this verifiable target, with clear configuration sensitivity including a near-zero or negative case. As an illustrative strongest-case example in a controlled instruction-versus-numeric setting, an instruction-tuned adapter substantially improves off-target NM-based numeric benchmark performance from 0.133 to 0.632 while not improving verifiable instruction following on IFEval (ILA: 0.313 to 0.271; PLA: 0.250 to 0.143; values rounded to three decimals). We refer to this nominal-versus-realized mismatch pattern as capability drift as a descriptive label. The mismatch is visible in the raw cross-task performance matrix; we use a drift score only as a compact summary in the same units as the underlying metrics, not as a new formal metric contribution. Evidence from broader instruction-following benchmarks is benchmark-dependent and mixed, reflecting heterogeneity in how instruction following is operationalized; we therefore do not treat cross-benchmark agreement as a premise. Overall, the practical takeaway is to perform routine cross-task evaluation before deployment and to avoid treating nominal labels as reliable capability proxies.
Figures & tables
Model
Nominal task
Numeric NM
IFEval ILA
IFEval PLA
Realized change
Base
–
0.133
0.313
0.250
Reference point.
Reason adapter
Numeric
0.309
0.271
0.179
Numeric gain aligned with target.
Instruction adapter
Instruction
0.632
0.271
0.143
Numeric gain, no IFEval gain.
Table 1: Illustrative strongest-case cross-task evaluation (main setting). We report NM-based numeric benchmark performance (NM) and IFEval strict, automatically verifiable instruction following (ILA/PLA), with values rounded to three decimals. This example provides a concrete view of the mismatch in raw cross-task metrics; robustness results are summarized in Table 2 and Appendix Table 6 .
Figure 1: Drift score summary. Heatmap of DriftScore(a→b) computed from cross-task evaluation metrics. Compact overview of score magnitudes from the same evaluations; see § 5 for context.
Figure 2: Robustness quadrant plot (primary evidence). Per-run target gain on IFEval PLA (instr − base) versus off-target gain on numeric NM (instr − base) for the robustness evaluations underlying Table 2 and Appendix Table 6 . Most points lie above y=x (positive drift), with occasional near-zero or negative cases.
Slice
Model
LoRA setting
n
Drift score ( ↑ )
Seeds
qwen3_8b
r16_attnmlp_do005_lrmain
5
0.511 ± 0.178
Seeds
qwen3_8b
r8_attnmlp_do005_lrmain
5
0.495 ± 0.150
Models
qwen3_14b
r16_attnmlp_do005_lrmain
2
0.667 ± 0.014
Models
qwen3_8b
r16_attnmlp_do005_lrmain
2
0.597 ± 0.010
Models
llama31_8b_instruct
r16_attnmlp_do005_lrmain
2
0.394 ± 0.016
Models
qwen25_7b_instruct
r16_attnonly_do0_lrsmall
1
-0.040
Table 2: Robustness summary of drift score (instr → numeric reasoning). Drift score remains positive across five seeds (Qwen3-8B) and across multiple base models and LoRA settings, with one model/setting showing near-zero or slightly negative drift (configuration sensitivity).
Figure 3: Selective shifts in verifiable instruction following (IFEval). Category-level and type-level shifts for the instruction adapter relative to base on IFEval. This breakdown is qualitative; low-support types should be interpreted cautiously.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Numeric EM
Numeric NM
IFEval ILA
IFEval PLA
Domain Acc
Domain MC
base
0.000
0.133
0.312
0.250
0.000
0.000
reason
0.042
0.306
0.271
0.179
0.000
0.000
instr
0.000
0.640
0.271
0.143
0.000
0.000
domain
0.000
0.281
0.345
0.148
0.000
0.000
Appendix
Table 3: Extended cross-task evaluation including a domain adapter (secondary). This table is provided for completeness and is not central to the paper’s primary robustness claim, which is established by Table 2 and Appendix Table 6 ; Table 1 is illustrative. All values are reported as fractions in [0,1] ; numeric benchmark performance is reported using both exact match (EM) and numeric match (NM).
Target metric
OffTargetGain (NM)
TargetGain (IFEval)
DriftScore (instr → numeric reasoning)
PLA
0.499
−0.107
0.606
ILA
0.499
−0.042
0.541
Appendix
Table 4: Target-metric sensitivity check (PLA vs ILA). Instruction → numeric reasoning drift score recomputed from Table 1 using IFEval PLA or ILA as the instruction target metric.
Model
IFBench PLA (strict) base
IFBench PLA (strict) instr
FollowBench rule mean base
FollowBench rule mean instr
Qwen3-8B
0.122
0.112
0.000
4.000
Qwen3-14B
0.143
0.116
0.000
2.000
Llama-3.1-8B-Instruct
0.139
0.119
4.000
4.500
Appendix
Table 5: Supplementary multi-benchmark instruction following. FollowBench rule-based mean and IFBench strict PLA for base vs instruction adapter. Values are computed from benchmark-suite outputs. FollowBench, IFBench, and IFEval differ in scale and emphasis, so absolute values should not be compared across benchmarks; in particular, FollowBench rule mean should be interpreted according to its scoring definition.
Figure 4: Upper-layer geometry diagnostics. Top modules by geometric similarity between instruction- and numeric-reasoning-tuned LoRA updates. This figure provides preliminary evidence but does not establish a localized causal mechanism.
Figure 5: Functional probing summary with null or small effects. Largest observed effect sizes from localized functional probing. We do not find strong localized causal evidence that explains the mismatch pattern in this setup.
Stage
Model
Setting
n
Drift score (mean ± std)
benchmark_suite
llama31_8b_instruct
r16_attnmlp_do005_lrmain
1
0.485
benchmark_suite
qwen3_14b
r16_attnmlp_do005_lrmain
1
0.674
benchmark_suite
qwen3_8b
r16_attnmlp_do005_lrmain
1
0.606
model_sweep
deepseek_r1_distill_qwen_7b
r16_attnonly_do0_lrsmall
1
0.284
model_sweep
deepseek_r1_distill_qwen_7b
r32_attnmlp_do005_lrmain
1
0.350
model_sweep
llama31_8b_instruct
r16_attnmlp_do005_lrmain
2
0.394±0.016
Appendix
Table 6: Robustness: drift score summary (instr → numeric reasoning). Drift score mean and standard deviation across seeds (when n>1 ); values are rounded to three decimals.