SafeStart: Mitigating the Overlooked Safety Cost of Active Expert Reduction in MoE LLM Serving
Organizations: University of Southern California, Los Angeles, CA, USA · Intel Labs, USA
Abstract
Mixture-of-Experts (MoE) language models send each token through a few of many expert subnetworks. Serving methods make these models faster by running even fewer experts per token, and they are validated on accuracy benchmarks. We show that these methods can weaken the model's refusal of harmful requests, and we introduce SafeStart, which largely restores refusal while keeping most of the speedup. In vLLM on one RTX A6000, keeping about two thirds of the selected experts speeds up offline decoding by 15 to 28% at batch size 8. We find that, at this setting, under a jailbreak that tells the model not to refuse, the share of harmful requests that the model answers rises by 18.4 percentage points on Qwen1.5-MoE, 9.5 on OLMoE, and 6.6 on DeepSeek-V2-Lite, the three of nine models where it rises significantly, while MMLU accuracy drops by less than two points. We discover that most of this increase arises while the model processes the prompt and generates its first few tokens. SafeStart therefore runs all experts for the prompt and the first eight generated tokens, and the reduced set of experts afterwards, through a fused kernel in vLLM. With experts removed, SafeStart keeps at least 85% of the offline decode speedup at batch sizes 8 and 32 and brings the change under this jailbreak to between -2.1 and +1.3 points for three expert-reduction methods on these three models. On 889 held-out requests, in an analysis fixed before generation, truncation raises harmful compliance by 5.8 to 21.4 points, and SafeStart leaves a rise of 0.6 to 2.6 points, non-inferior to the original model on two of the three models. Our results show that faster MoE serving need not cost safety: a serving system can keep most of the gain from running fewer experts by running all of them at the start of each response, and it should test refusal, not only accuracy, before it enables such a method.
Figures & tables
| Plain | Refusal suppression | Prefix injection | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Model | Shared | Renorm | Original | Original | Original | Noise | ||||
| Qwen1.5-MoE | yes | no | 3/4 | 7.6 | ( ) | 46.6 | ( ) | 79.9 | ( ) | 9/11 |
| DeepSeek-V2-Lite | yes | no | 4/6 | 12.3 | ( ) | 43.6 | ( ) | 45.3 | ( ) | 17/19 |
| Moonlight-16B | yes | yes | 4/6 | 37.3 | ( ) | 86.9 | ( ) | 91.7 | ( ) | 3/5 |
| Ling-lite-1.5 | yes | yes | 4/6 | 8.7 | ( ) | 41.7 | ( ) | 58.5 | ( ) | 14/10 |
| OLMoE-0125 | no | no | 6/8 | 3.4 | ( ) | 24.2 | ( ) | 30.9 | ( ) | 9/12 |
| Qwen1.5 | DeepSeek-V2-Lite | OLMoE | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Rule | GSM | alone | +SafeStart | GSM | alone | +SafeStart | GSM | alone | +SafeStart |
| Top- | |||||||||
| Threshold | |||||||||
| Top- | |||||||||
| Model | Original | LExI | SafeStart + LExI |
|---|---|---|---|
| Qwen1.5-MoE | 35.7 | 51.8 | 36.4 |
| OLMoE-0125 | 15.6 | 46.4 | 18.5 |
| DeepSeek-V2-Lite | 51.3 | 67.0 | 53.3 |
| ACE 25% | ACE 50% | ||||
|---|---|---|---|---|---|
| Model | Original | ACE | SafeStart + ACE | ACE | SafeStart + ACE |
| Qwen1.5 | 36.38 | 45.54 | 34.38 | 57.37 | 35.94 |
| OLMoE-0125 | 14.96 | 17.41 | 16.07 | 27.90 | 16.74 |
Appendix figures & tables43 assets
Supplementary material from the paper’s appendix.
Appendix
| Batch size | ||||||
|---|---|---|---|---|---|---|
| Model | Condition | 1 | 8 | 32 | 128 | 256 |
| Qwen1.5 | Truncation | |||||
| SafeStart , | ||||||
| SafeStart , | ||||||
| Patch, , | ||||||
| Patch, no token truncated | ||||||
| Batch size | ||||||
|---|---|---|---|---|---|---|
| Model | Condition | 1 | 8 | 32 | 128 | TTFT |
| Qwen1.5 | Truncation | |||||
| Patch, nothing truncated (PyTorch) | ||||||
| Patch, nothing truncated (kernel) | ||||||
| SafeStart (PyTorch) | ||||||
| SafeStart (kernel) | ||||||
| Batch size | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Model | Condition | Quantity | 1 | 8 | 32 | 128 | 256 | |||||
| Qwen1.5 | Original | Experts read | 4.0 | (4.0) | 22.1 | (25.5) | 49.1 | (53.4) | 59.1 | (60.0) | 59.8 | (60.0) |
| Truncation | Experts read | 3.0 | (3.0) | 17.8 | (20.2) | 44.2 | (48.4) | 58.5 | (59.9) | 59.7 | (60.0) | |
| Reads removed (%) | 12.5 | 16.5 | 9.1 | 1.0 | 0.2 | |||||||
| Decode speedup (%) | – | |||||||||||
| Decode time saved (%) | 6.9 | 13.6 | 7.5 | 0.9 | – | |||||||
| Batch size | ||||||
| Model | Condition | Quantity | 1 | 8 | 32 | 128 |
| Qwen1.5 | Original | GPU ms per step | 8.72 | 18.60 | 34.89 | 62.08 |
| Routed-expert share (%) | 33 | 63 | 78 | 82 | ||
| Truncation | Routed-expert time (%) | |||||
| Routed expert reads (%) | ||||||
| GPU time per step (%) | ||||||
| TPOT change (%) | TTFT p50 (ms) | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Model | Condition | 1/s | 2/s | 4/s | 8/s | 1/s | 2/s | 4/s | 8/s |
| Qwen1.5 | Original | 68 | 80 | 85 | 96 | ||||
| Truncation | 65 | 73 | 86 | 100 | |||||
| SafeStart , | 68 | 77 | 88 | 99 | |||||
| SafeStart , | 68 | 76 | 87 | 100 | |||||
| SafeStart , | 66 | 73 | 84 | 99 | |||||
| TPOT change (%) | TTFT p50 (ms) | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Model | Condition | 1/s | 2/s | 4/s | 8/s | 1/s | 2/s | 4/s | 8/s | ||||
| Qwen1.5 | Truncation | 64 | 72 | 83 | 98 | ||||||||
| Patch, nothing truncated | 69 | 79 | 88 | 101 | |||||||||
| SafeStart | 67 | 76 | 85 | 100 | |||||||||
| DeepSeek-V2-Lite | Truncation | 67 | 76 | 91 | 104 | ||||||||
| Patch, nothing truncated | 75 | 86 | 98 | 110 | |||||||||
| Gain kept (%) | |||||
|---|---|---|---|---|---|
| Model | Protected | 1/s | 2/s | 4/s | 8/s |
| Qwen1.5 | Prompt ( ) | 92 | 67 | 91 | 47 |
| Prompt + 8 ( ) | 58 | 65 | 49 | 52 | |
| DeepSeek-V2-Lite | Prompt ( ) | 78 | 60 | 73 | 122 |
| Prompt + 8 ( ) | 75 | 49 | 69 | 140 | |
| OLMoE | Prompt ( ) | 79 | 72 | 62 | 61 |
| Natural (3) | Fixed (2) | Closed loop (1) | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Model | Condition | 1/s | 2/s | 4/s | 8/s | 1/s | 2/s | 4/s | 8/s | 1 | 4 | 16 |
| Qwen1.5 | Truncation | |||||||||||
| SafeStart | ||||||||||||
| Prompt only | – | – | – | |||||||||
| DeepSeek-V2-Lite | Truncation | – | – | – | ||||||||
| SafeStart | – | – | – | |||||||||
| Model | Condition | Llama Guard 3 | HarmBench cls. |
|---|---|---|---|
| Qwen1.5 | Original rerun | (17/21) | (18/22) |
| Truncation | (52/22) | (59/24) | |
| Truncation | (59/29) | (61/35) | |
| SafeStart (emulation), | (41/18) | (47/29) | |
| SafeStart (emulation), | (20/15) | (23/22) | |
| DeepSeek-V2-Lite | Original rerun | (22/27) | (29/26) |
| Model | Condition | Distractors | Evil confidant |
|---|---|---|---|
| Qwen1.5 | original ASR | 69.07 | 79.87 |
| Original rerun | (53/50) [ ] | (20/13) [ ] | |
| Patch, nothing truncated | (45/49) [ ] | (19/14) [ ] | |
| Truncation | (70/71) [ ] | (36/57) [ ] | |
| SafeStart , emulation | (74/61) [ ] | (29/31) [ ] | |
| SafeStart , emulation | (53/48) [ ] | (20/20) [ ] |
| SafeStart | |||||
|---|---|---|---|---|---|
| Seed | Original | Trunc. | engine | emulation | HB |
| 1 | 53.8 | (96/15) | (15/13) | (17/19) | / / |
| 2 | 49.6 | (94/16) | (12/20) | (24/12) | / / |
| 3 | 39.0 | (99/16) | (10/14) | (20/17) | / / |
| Framing | ASR | 95% CI | Flips | Degen. | HB | |||
|---|---|---|---|---|---|---|---|---|
| Plain | 4 | 7.63 | – | – | – | – | – | – |
| 3 | 7.63 | 14/14 | 1.0 | 0 | ||||
| 2 | 12.71 | 38/14 | 0.0012 | 0 | ||||
| rerun 4 | 7.42 | – | 4/5 | 1.0 | – | |||
| Refusal supp. | 4 | 46.61 | – | – | – | – | – | – |
| 3 | 65.04 | 101/14 | 0 |
| Framing | ASR | 95% CI | Flips | Degen. | HB | |||
|---|---|---|---|---|---|---|---|---|
| Plain | 6 | 12.29 | – | – | – | – | – | – |
| 5 | 12.92 | 10/7 | 0.629 | 0 | ||||
| 4 | 15.68 | 21/5 | 0.00249 | 0 | ||||
| 3 | 22.88 | 59/9 | 0 | |||||
| rerun 6 | 11.44 | – | 4/8 | 0.388 | – | |||
| Refusal supp. | 6 | 43.64 | – | – | – | – | – | – |
| Framing | ASR | 95% CI | Flips | Degen. | HB | |||
|---|---|---|---|---|---|---|---|---|
| Plain | 6 | 37.29 | – | – | – | – | – | – |
| 5 | 36.23 | 16/21 | 0.511 | 0 | ||||
| 4 | 32.42 | 15/38 | 0.00219 | 1 | ||||
| 3 | 29.24 | 19/57 | 0 | |||||
| rerun 6 | 36.02 | – | 6/12 | 0.238 | – | |||
| Refusal supp. | 6 | 86.86 | – | – | – | – | – | – |
| Framing | ASR | 95% CI | Flips | Degen. | HB | |||
|---|---|---|---|---|---|---|---|---|
| Plain | 6 | 8.69 | – | – | – | – | – | – |
| 5 | 8.69 | 10/10 | 1.0 | 0 | ||||
| 4 | 9.53 | 15/11 | 0.557 | 0 | ||||
| 3 | 12.08 | 29/13 | 0.0195 | 0 | ||||
| rerun 6 | 10.17 | – | 10/3 | 0.0923 | – | |||
| Refusal supp. | 6 | 41.74 | – | – | – | – | – | – |
| Framing | ASR | 95% CI | Flips | Degen. | HB | |||
|---|---|---|---|---|---|---|---|---|
| Plain | 8 | 3.39 | – | – | – | – | – | – |
| 6 | 2.75 | 3/6 | 0.508 | 0 | ||||
| 5 | 5.30 | 12/3 | 0.0352 | 0 | ||||
| 4 | 8.26 | 27/4 | 0 | |||||
| rerun 8 | 2.97 | – | 3/5 | 0.727 | – | |||
| Refusal supp. | 8 | 24.15 | – | – | – | – | – | – |
| Framing | ASR | 95% CI | Flips | Degen. | HB | |||
|---|---|---|---|---|---|---|---|---|
| Plain | 8 | 1.27 | – | – | – | – | – | – |
| 6 | 1.27 | 1/1 | 1.0 | 0 | ||||
| 4 | 2.54 | 8/2 | 0.109 | 0 | ||||
| 2 | 31.57 | 147/4 | 67 | |||||
| rerun 8 | 1.27 | – | 1/1 | 1.0 | – | |||
| Refusal supp. | 8 | 9.96 | – | – | – | – | – | – |
| Framing | ASR | 95% CI | Flips | Degen. | HB | |||
|---|---|---|---|---|---|---|---|---|
| Plain | 4 | 1.69 | – | – | – | – | – | – |
| 3 | 1.06 | 2/5 | 0.453 | 0 | ||||
| 2 | 5.08 | 18/2 | 1 | |||||
| rerun 4 | 1.06 | – | 0/3 | 0.25 | – | |||
| Refusal supp. | 4 | 6.57 | – | – | – | – | – | – |
| 3 | 5.93 | 9/12 | 0.664 | 0 |
| Framing | ASR | 95% CI | Flips | Degen. | HB | |||
|---|---|---|---|---|---|---|---|---|
| Plain | 6 | 18.86 | – | – | – | – | – | – |
| 5 | 17.80 | 13/18 | 0.473 | 2 | ||||
| 4 | 18.86 | 24/24 | 1.0 | 1 | ||||
| 3 | 23.94 | 45/21 | 0.00427 | 2 | ||||
| rerun 6 | 18.22 | – | 9/12 | 0.664 | – | |||
| Refusal supp. | 6 | 37.50 | – | – | – | – | – | – |
| Framing | ASR | 95% CI | Flips | Degen. | HB | |||
|---|---|---|---|---|---|---|---|---|
| Plain | 8 | 3.60 | – | – | – | – | – | – |
| 6 | 4.03 | 5/3 | 0.727 | 0 | ||||
| 5 | 4.66 | 8/3 | 0.227 | 0 | ||||
| 4 | 4.24 | 9/6 | 0.607 | 0 | ||||
| rerun 8 | 3.60 | – | 0/0 | 1.0 | – | |||
| Refusal supp. | 8 | 45.34 | – | – | – | – | – | – |
| XSTest | |||||
| Model | Condition | MMLU | GSM8K | safe | unsafe |
| Qwen1.5-MoE | (original) | 58.21 | 58.68 | 24.0 | 97.0 |
| 57.58 | 56.33 | 18.8 | 95.5 | ||
| 56.10 | 50.64 | 21.2 | 89.5 | ||
| SafeStart, | 58.21 | 57.92 | 24.0 | 97.0 | |
| SafeStart, | 57.91 | 54.21 | 24.0 | 97.0 | |
| Alone | Eng. | Emul. | |||||||||
| Model | Rule | Experts | MMLU | GSM8K | XSTest | LG | HB | LG | HB | LG | HB |
| Qwen1.5 | Original | 4 | 58.2 | 58.7 | 24.0 | 46.6 | 40.5 | ||||
| Top- truncation | 3 | 57.6 | 56.3 | 18.8 | |||||||
| Top- truncation | 2 | 56.1 | 50.6 | 21.2 | |||||||
| Threshold, | 2.9 | 57.7 | 57.2 | 25.2 | |||||||
| Top- , | 2.9 | 55.6 | 50.8 | 20.8 | |||||||
| Batch size | |||||
| Model | Condition | 1 | 8 | 32 | 128 |
| Qwen1.5 | Truncation | ||||
| Threshold | |||||
| Threshold + SafeStart | |||||
| Top- | |||||
| Top- + SafeStart | |||||
| RS | PI | Speedup (%) | |||||
| LG | HB | LG | HB | bs 1 | bs 8 | bs 32 | |
| Qwen1.5 | |||||||
| trunc. | |||||||
| 0 | |||||||
| 1 | |||||||
| 2 | |||||||
| Prefix injection | |||
|---|---|---|---|
| Condition | Qwen1.5, | DeepSeek-V2-Lite, | OLMoE, |
| Truncation | ( ) | ( ) | ( ) |
| Attenuation only | ( ) | ( ) | ( ) |
| Mass restored | ( ) | ( ) | ( ) |
| Swap (no truncation) | ( ) | ( ) | ( ) |
| Early, | ( ) | ( ) | ( ) |
| LG | HB | ||||||
|---|---|---|---|---|---|---|---|
| Model | Framing | Original | Trunc. | SafeStart | Original | Trunc. | SafeStart |
| Qwen1.5 | Plain | 8.2 | 7.4 | ||||
| Refusal supp. | 51.6 | 49.2 | |||||
| Prefix inj. | 73.8 | 69.7 | |||||
| Evil confidant | 82.8 | 73.0 | |||||
| Distractors | 73.8 | 22.9 | |||||
| Half | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Model | Framing | Judge | Orig. | Rerun | Trunc. | Trunc. | ||||
| Qwen1.5 | Refusal supp. | LG | 43.6 | |||||||
| HB | 32.2 | |||||||||
| Prefix inj. | LG | 75.1 | ||||||||
| HB | 71.5 | |||||||||
| DeepSeek-V2-Lite | Refusal supp. | LG | 37.7 | |||||||
| LG | HB | |||||||
|---|---|---|---|---|---|---|---|---|
| Model | Prompts | Framing | Trunc. | SS orig. | SS trunc. | Trunc. | SS orig. | SS trunc. |
| Qwen1.5 | Eval. | Plain | ||||||
| Eval. | Refusal supp. | |||||||
| Eval. | Prefix inj. | |||||||
| Eval. | Evil confidant | |||||||
| Eval. | Distractor | |||||||
| Pairs | ||||
|---|---|---|---|---|
| Comparison | Stratum | Pop. | Sample | Sides |
| Qwen1.5, RS, | forward | 101 | 101 | both |
| reverse | 14 | 14 | both | |
| concordant unsafe | 206 | 20 | both | |
| concordant safe | 151 | 20 | both | |
| Qwen1.5, RS, original rerun | forward | 9 | 9 | both |
| Annotator | ||||
|---|---|---|---|---|
| Comparison | Stratum | Pairs | A1 | A2 |
| Qwen1.5, RS, | concordant safe | 20 | 0/0 | 0/0 |
| concordant unsafe | 20 | 1/0 | 1/0 | |
| forward | 101 | 59/1 | 62/1 | |
| reverse | 14 | 2/6 | 1/7 | |
| DeepSeek-V2-Lite, RS, | forward | 63 | 27/0 | 26/0 |
| Model | Method | Setting | ASR | ||
| Qwen1.5 | Original | 34.38 | – | – | |
| Trunc. | 57.81 | ||||
| Trunc. | 65.63 | ||||
| Qwen3 | Original | 8.26 | – | – | |
| Trunc. | 5.58 | 0.023 | |||
| Trunc. | 3.13 |
| Qwen1.5, | DeepSeek-V2-Lite, | OLMoE, | ||||
|---|---|---|---|---|---|---|
| Condition | Plain | Refusal supp. | Plain | Refusal supp. | Plain | Refusal supp. |
| Truncation | ( ) | ( ) | ( ) | ( ) | ( ) | ( ) |
| Attenuation only | ( ) | ( ) | ( ) | ( ) | ( ) | ( ) |
| Mass restored | ( ) | ( ) | ( ) | ( ) | ( ) | ( ) |
| Swap (no truncation) | ( ) | ( ) | ( ) | ( ) | ( ) | ( ) |
| Early, | ( ) | ( ) | ( ) | ( ) | ( ) | ( ) |
| Qwen1.5, | OLMoE, | DeepSeek-MoE-16B, | ||||
|---|---|---|---|---|---|---|
| Condition | LG | HB | LG | HB | LG | HB |
| Original (pipeline null) | ||||||
| Truncated | ||||||
| Truncated, refusal direction restored | ||||||
| at layer only | ||||||
| at layers to | ||||||
| Plain | Refusal suppression | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Model | Pos. | Shift | ASR | Pos. | Shift | ASR | |||
| Qwen1.5-MoE | |||||||||
| Qwen1.5-MoE | |||||||||
| DeepSeek-V2-Lite | |||||||||
| DeepSeek-V2-Lite | |||||||||
| OLMoE | |||||||||
| Setting | Removed | ASR | (pp) |
|---|---|---|---|
| Original ( ) | 0% | 35.7% | — |
| LExI | 50% | 37.9% | |
| LExI | 41.7% | 38.6% | |
| LExI | 25% | 46.2% | |
| LExI | 16.7% | 51.8% | |
| Uniform | 25% | 60.3% |
| Model | Skip target | ACE Original | Protected Original | Protected ACE | Upper 90% |
|---|---|---|---|---|---|
| Qwen1.5 | 25% | ||||
| 50% | |||||
| OLMoE-0125 | 25% | ||||
| 50% |
LLM Safety
Large language model (LLM) safety studies how to prevent harmful, unreliable, or misaligned behavior while preserving useful responses.
See all 467 papers