Understanding a language model's decision requires knowing how its internal computations affect its preference between answers. Gradient-based methods estimate this influence by predicting the effect of changing those computations. In mixture-of-experts (MoE) models, replacing one expert can change which experts later layers select. Those switches can alter the answer scores in ways that derivatives do not capture. We introduce a predictor that uses the model's own expert outputs and selection rules to estimate these extra effects, then adds them to a second-order prediction. Across questions from three benchmarks, our method reduces the error in predicting answer-score changes by 60% on Qwen3.6 and 47% on Gemma4, relative to a multistep baseline using gradients and second derivatives.
Figures & tables
Model
Dataset
AtP
HVP1
HVP5
Ours
Reduction
HVP5 − ours (mean ± SE)
Qwen3.6
MMLU-Pro
0.0676
0.0629
0.0626
0.0250
60.0%
0.0375±0.0084
SuperGPQA
0.0557
0.0521
0.0500
0.0185
63.0%
0.0315±0.0055
HLE
0.0357
0.0334
0.0327
0.0125
61.7%
0.0202±0.0029
Gemma4
MMLU-Pro
0.1478
0.1209
0.1144
0.0602
47.4%
0.0542±0.0111
SuperGPQA
0.0975
0.0790
0.0722
0.0412
43.0%
0.0310±0.0053
HLE
0.1031
0.0836
0.0653
0.0507
22.4%
0.0146±0.0094
Table 1: Predicting changes in answer preference. MAE in logit units; lower is better. All methods use the same 13 interventions per question. Reduction is relative to HVP5; the last column is HVP5 MAE minus ours. Full comparisons and per-question observations are in Appendix D .
Qwen3.6
Gemma4
Predictor
Recall
Mass
Recall
Mass
Predictors without measured single effects
AtP
54.2
79.3
62.5
85.3
HVP5
63.3
84.3
78.3
93.5
Ours
79.2
93.1
83.3
97.5
Ours − HVP5
15.8±6.5
8.8±4.4
5.0±4.5
4.0±2.1
Table 2: Selecting combinations with the largest effects on answer scores. Recall@5 and captured effect mass (%) among 32 joint candidates. Difference rows are Ours minus the named comparator, reported as paired mean ± SE in percentage points, calculated before rounding. Single sum adds measured single-replacement effects; the predictors above do not use these measurements.
Figure 1: Later expert choices are a source of prediction error. (a) Keeping the same experts selected sharply reduces HVP5 error; segments pair the same question. (b) Most mean effect magnitude remains. Dots show question-level ratios; diamonds show ratios of model means. (c) Increasing HVP from five to twenty steps barely helps on the same eight joint cases. Small points are questions, large markers are means. Each model uses 24 questions; (a,b) use all 32 joint candidates.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Qwen3.6
Gemma4
Predictor
MMLU-Pro
SuperGPQA
HLE
MMLU-Pro
SuperGPQA
HLE
Zero change
0.1223
0.0990
0.0768
0.3001
0.2264
0.2757
AtP
0.0676
0.0557
0.0357
0.1478
0.0975
0.1031
IG5
0.0629
0.0506
0.0328
0.1162
0.0743
0.0687
HVP1
0.0629
0.0521
0.0334
0.1209
0.0790
0.0836
HVP5
0.0626
0.0500
0.0327
0.1144
0.0722
0.0653
Appendix
Table 3: Prediction and component comparisons. MAE in logit units on the same 13 interventions per question. Δ MAE is HVP5 minus ours; positive values favor our predictor. The last rows give the relative reduction, paired mean ± SE, and number of questions with lower error. Component variants use the same natural-routing target.
Figure 2: Paired prediction errors across all evaluation questions. Each point gives HVP5 MAE (horizontal) and our MAE (vertical) for one question, averaged over its 13 matched interventions. Points below the dashed equality line favor our predictor. Both axes are logarithmic, with identical limits across panels. All 158 questions per model are shown; symbols and colors identify datasets. Paired uncertainty is reported in Table 3 .
Figure 3: Question-level evidence for the effect of expert selection. (a) HVP5 error under natural routing (circles) and fixed later expert sets (squares); vertical segments join the same question. (b) Fixed-to-natural ratios of mean absolute effect: dots are per-question ratios and diamonds are ratios of the model-level means. (c) HVP5, HVP20, and our predictor on the same eight joint conditions: small points are questions and large markers are means. Panels (a,b) use 32 joint conditions. Each model has 24 questions; horizontal offsets separate observations without changing their values.
Qwen3.6
Gemma4
Measure
Natural
Fixed
Natural
Fixed
HVP5 MAE
0.055332
0.000249
0.121509
0.001385
Mean ∣Δ∣
0.1078
0.0983
0.4007
0.3718
Normalized MAE
0.5313
0.0026
0.3389
0.0034
Recall@5 (%)
63.3
100.0
78.3
98.3
Model-level routing summaries
Appendix
Table 4: Native routing controls and observed routing changes. Natural and fixed executions use the same source assignments. Normalized MAE averages each question’s error divided by its mean absolute effect. The lower block summarizes each model: changed sets and layers refer to natural routing; retained effect is the ratio of fixed to natural mean absolute effects, not a mediated-effect fraction. Recall is in percent.
Figure 4: Paired gains in identifying influential combinations. Points show paired mean gains, and error bars show ± one SE. Positive values favor our predictor; the dashed line denotes equal performance. All comparisons use 24 questions per model and the same 32 joint candidates. The single-sum reference uses measured single-intervention effects. Gemma’s recall gain against HVP5 is comparable to its SE.
Qwen3.6
Gemma4
Predictor
MAE
Recall@5
Mass
MAE
Recall@5
Mass
Predictors without measured single effects
AtP
0.0687
54.2
79.3
0.1741
62.5
85.3
HVP1
0.0579
65.8
85.2
0.1260
71.7
90.1
HVP5
0.0553
63.3
84.3
0.1215
78.3
93.5
Full response
0.0396
71.7
89.1
0.1256
65.0
87.9
Appendix
Table 5: Joint-intervention accuracy, ranking, and paired uncertainty. Recall@5 and effect-mass capture are percentages; MAE is in logit units. Each gain row names its comparator and reports a paired mean ± SE. For MAE, gain is comparator minus ours; for ranking, it is ours minus comparator in percentage points. Differences use values before rounding. The separate single-sum reference adds measured constituent effects.
Recall@5 (%)
Effect mass captured (%)
Dataset
HVP5
Ours
Ours − HVP5
HVP5
Ours
Ours − HVP5
Qwen3.6
MMLU-Pro
73.9
90.4
16.4±2.2
90.3
97.4
7.1±1.4
SuperGPQA
74.2
87.3
13.1±2.7
88.5
97.1
8.7±1.8
HLE
72.0
88.8
16.8±2.5
87.9
98.1
10.2±1.4
Gemma4
Appendix
Table 6: Ranking on the complete question collection. Recall and captured effect mass on the 13-candidate panel. Gains are ours minus HVP5, in percentage points, reported as mean ± SE. Each model uses the same 56 MMLU-Pro, 52 SuperGPQA, and 50 HLE questions; the methods use identical candidates.
Model
Compile
HVP1
HVP5
Forward
Qwen3.6
67.3
44.3
219.7
2.33
Gemma4
68.9
10.7
52.4
2.10
Appendix
Table 7: Recorded computation costs in seconds. Compile is a per-prompt cost. HVP1, HVP5, and direct Forward cover all 32 joint candidates. Means are from the A100 80GB study.
Large language model (LLM) interpretability studies how internal representations and computations produce outputs, and how they can be explained or controlled.
Mixture-of-Experts (MoE) models reduce computation through sparse expert activation, but deployment still requires storing a large number of expert parameters. We study post-training expert pruning using task-domain calibration data as a subset-selection problem: given a pretrained MoE layer, select a fixed number of experts that best reconstruct its original output. Exact subset search is combinatorial and becomes intractable for modern MoE models with many experts per layer. We develop OptEP, a scalable continuous relaxation that directly optimizes differentiable expert-selection variables, followed by a lightweight reconstruction-based update of the retained expert weights. Across multiple MoE architectures and tasks spanning mathematical reasoning and code generation, our approach improves over the strongest existing expert-pruning baselines, with absolute accuracy gains of up to 15.05% on Qwen3-30B-A3B, 14.55% on Qwen3-235B-A22B, 11.06% on GLM-4.5-Air, and 16.20% on Mixtral-8x7B. The improvements are particularly pronounced under aggressive pruning, demonstrating that reconstruction-driven expert selection and weight adaptation can preserve substantially more of the original model's performance while reducing the number of retained experts.
Jelena Markovic-Voronov, Petros Prastakos, Kayhan Behdin +7
Sparse Mixture-of-Experts (MoE) architectures enable scaling LLM parameters under a fixed inference budget by activating only a small subset of experts via top-k routing. While this preserves causality and suits autoregressive language models, the discrete top-k operator is not differentiable, forcing a fixed number of active experts per input and resulting in inefficient use of computation. We propose SoftMoE, which replaces discrete routing with a truncated soft top-k LapSum relaxation, allowing gradient-based optimization of expert routing. We further parameterize the mean number of active experts per layer and impose a global budget constraint, enabling the model to learn how to allocate expert capacity across layers. SoftMoE remains fully compatible with autoregressive modeling and achieves performance comparable to or better than sparse MoE on language modeling and downstream tasks, while activating significantly fewer experts. Notably, the learned allocation is highly non-uniform, with later layers activating more experts. The source code is publicly available†.
Mikołaj Zasada, Łukasz Struski, Jacek Tabor +1
AGH University of Krakow, Poland · Faculty of Mathematics and Computer Science, Jagiellonian University, Poland · Jagiellonian University, Poland +1
Mixture-of-Experts (MoE) models have gained popularity as a means of scaling the capacity of large language models (LLMs) while maintaining sparse activations and reduced per-token compute. However, in memory-constrained inference settings, expert weights must be offloaded to CPU, creating a performance bottleneck from CPU-GPU transfers during decoding. We propose an expert prefetching scheme that leverages currently computed internal model representations to speculate future experts, enabling memory transfers to overlap with computation. Across multiple MoE architectures, we demonstrate that future experts can be reliably predicted by these internal representations. We also demonstrate that executing speculated experts generally maintains downstream task accuracy, thus preserving more effective compute-memory overlap by eliminating the need to re-fetch true router-selected experts. Integrated into an optimized inference engine, our approach achieves up to 14% reduction in time per output token (TPOT) over on-demand loading of experts from CPU memory. For MoEs where speculative execution alone yields suboptimal accuracy, we further examine lightweight estimators that improve expert prediction hit rates, thereby reducing performance degradation. Our code is released in open-source at https://github.com/axonn-ai/yalis/tree/offload_prefetch.