Predict, Don't React: Value-Based Safety Forecasting for LLM Streaming

Pride Kavumba; Koki Wataoka; Huy H. Nguyen; Jiaxuan Li; Masaya Ohagi

cs.CLApr 5, 2026

Predict, Don't React: Value-Based Safety Forecasting for LLM Streaming

Authors: Pride Kavumba, Koki Wataoka, Huy H. Nguyen, Jiaxuan Li, Masaya Ohagi

Organizations: SB Intuitions Corp.

Abstract

In many practical LLM deployments, a single guardrail is used for both prompt and response moderation. Prompt moderation operates on fully observed text, whereas streaming response moderation requires safety decisions to be made over partial generations. Existing text-based streaming guardrails commonly frame this output-side problem as boundary detection, training models to identify the earliest prefix at which a response has already become unsafe. In this work, we introduce StreamGuard, a unified model-agnostic streaming guardrail that instead formulates moderation as a forecasting problem: given a partial prefix, the model predicts the expected harmfulness of likely future continuations. We supervise this prediction using Monte Carlo rollouts, which enables early intervention without requiring exact token-level boundary annotations. Across standard safety benchmarks, StreamGuard performs strongly both for input moderation and for streaming output moderation. At the 8B scale, StreamGuard improves aggregated input-moderation F1 from 86.7 to 88.2 and aggregated streaming output-moderation F1 from 80.4 to 81.9 relative to Qwen3Guard-Stream-8B-strict. On the QWENGUARDTEST response_loc streaming benchmark, StreamGuard reaches 97.5 F1, 95.1 recall, and 92.6% on-time intervention, compared to 95.9 F1, 92.1 recall, and 89.9% for Qwen3Guard-Stream-8B-stric, while reducing the miss rate from 7.9% to 4.9%. We further show that forecasting-based supervision transfers effectively across tokenizers and model families: with transferred targets, Gemma3-StreamGuard-1B reaches 81.3 response-moderation F1, 98.2 streaming F1, and a 3.5% miss rate. These results show that strong end-to-end streaming moderation can be obtained without exact boundary labels, and that forecasting future risk is an effective supervision strategy for low-latency safety intervention.

Figures & tables

Figure 1: Boundary detection versus forecasting in streaming moderation. Top: Boundary-detection methods evaluate whether the current prefix has already crossed an unsafe boundary, so intervention is tied to when unsafe content becomes explicit in the text. Bottom: StreamGuard instead scores each prefix by its expected future harmfulness , predicting whether likely continuations are likely to lead to an unsafe completion. This supports early intervention in streaming settings without requiring exact token-level boundary annotations.

Model	Streaming	ToxiC	OAIMod	Aegis	Aegis2	SSTest	HarmB	WildG	Avg
PolyGuard-Qwen-7B ( Kumar et al., 2025 )	no	71.5	74.1	90.3	86.3	100.0	98.7	88.1	87.0
Qwen3Guard-Stream-0.6B-strict	yes	72.0	68.3	85.2	84.9	98.0	97.2	87.1	84.7
Qwen3Guard-Stream-0.6B-loose	yes	75.5	76.0	77.7	81.7	96.9	96.8	86.0	84.4
Qwen3Guard-Stream-4B-strict	yes	73.0	70.0	85.9	86.6	99.5	100.0	88.6	86.2
Qwen3Guard-Stream-4B-loose	yes	81.7	81.2	75.5	80.2	98.5	98.9	85.3	85.9
Qwen3Guard-Stream-8B-strict	yes	75.3	74.0	85.7	86.1	99.0	99.4	87.5	86.7

Table 1: Input-moderation F1; Avg denotes the macro-average. For prior systems, we report published results. We report mean

\pm

std over three seeds for StreamGuard models. The Streaming column indicates whether a model supports token-level moderation during generation. For brevity, we show only the strongest offline post-hoc model by macro F1 as a reference; full offline results appear in Appendix B .

Model	Streaming	HarmB	SRLHF	BeaverTails	XSTestR	Aegis2	WildG	Avg
Qwen3Guard-4B-Gen-strict	no	86.7	69.8	86.6	92.7	86.1	79.5	83.6
Qwen3Guard-8B-Gen-strict	no	87.2	70.5	86.6	92.1	86.1	78.9	83.6
Qwen3Guard-Stream-0.6B-strict	yes	83.1	62.8	84.5	84.8	81.4	76.3	78.8
Qwen3Guard-Stream-0.6B-loose	yes	80.6	61.7	84.0	83.3	81.4	75.8	77.8
Qwen3Guard-Stream-4B-strict	yes	84.3	67.6	86.0	88.5	83.1	76.4	81.0
Qwen3Guard-Stream-4B-loose	yes	83.6	64.3	85.2	88.9	83.3	77.4	80.5

Table 2: Output-moderation F1; Avg denotes the macro-average. For prior systems, we report published results. StreamGuard models are evaluated in our simulated streaming protocol, processing responses one token at a time and counting a response as unsafe if it triggers under the debounced rule in § 3.1 . We report mean

\pm

Model	QwenGuardTest				XSTestR
Model	F1	OnTime (%)	Late (%)	Miss (%)	F1	FPR
Qwen3Guard-Stream-0.6B-strict	95.0	88.3	2.1	9.6	83.8	7.1
Qwen3Guard-Stream-0.6B-loose	94.3	87.9	1.2	10.8	83.8	7.1
Llama3-StreamGuard-1B ( ours )	97.2 $\pm\,0.2$	92.9 $\pm\,0.5$	1.6 $\pm\,0.2$	5.5 $\pm\,0.4$	84.6 $\pm{0.7}$	6.3 $\pm{0.4}$
Qwen3Guard-Stream-4B-strict	96.8	91.0	2.7	6.3	87.7	4.9
Qwen3Guard-Stream-4B-loose	94.9	88.3	2.0	9.7	88.0	5.4

Table 3: Streaming safety evaluation on QwenGuardTest and overblocking on XSTest-Resp . Interventions are categorized as OnTime (at or before the unsafe sentence ends), Late (after the sentence ends), or Miss (never triggered) (§ 3.1 ). XSTest-Resp reports response F1 and benign FPR. StreamGuard models report mean

\pm

std over three seeds.

Model	X-Tok	Input Mod	Output Mod	QwenGuardTest				XSTestR
Model	X-Tok	F1 Mean	F1 Mean	F1	OnTime (%)	Late (%)	Miss (%)	F1	FPR
Qwen3Guard-Stream-0.6B-strict	No	84.7	78.8	95.0	88.3	2.1	9.6	83.8	7.1
Qwen3Guard-Stream-0.6B-loose	No	84.4	77.8	94.3	87.9	1.2	10.8	83.8	7.1
Llama3-StreamGuard-1B	No	86.4	80.2	97.2 $\pm 0.2$	92.9 $\pm 0.5$	1.6 $\pm 0.2$	5.5 $\pm 0.4$	84.6 $\pm 0.7$	6.3 $\pm 0.4$
Gemma3-StreamGuard-0.3B	Yes	79.9	75.3	97.6 $\pm 0.4$	91.3 $\pm 0.4$	4.0 $\pm 0.4$	4.8 $\pm 0.8$	76.1 $\pm 2.4$	11.6 $\pm 1.2$
Gemma3-StreamGuard-1B	Yes	85.1	81.3	98.2 $\pm 0.2$	92.3 $\pm 0.3$	4.2 $\pm 0.1$	3.5 $\pm 0.4$	87.2 $\pm 0.0$	5.2 $\pm 0.0$

Table 4: Transfer across tokenizers and backbone families. X-Tok denotes prefix-level supervision transferred from a source with a different tokenizer. Input and Output Mod report macro-averaged F1 across standard suites (dataset-level breakdowns in Appendix D ). QwenGuardTest and XSTest-Resp protocols follow § 3.1 . StreamGuard models report mean

\pm

std over three seeds.

Factor	Variant	Input Mod	Output Mod	QwenGuardTest				XSTestR
Factor	Variant	F1 Mean	F1 Mean	F1	OnTime (%)	Late (%)	Miss (%)	F1	FPR
Reference	Mean Reduction	88.2	81.9	97.5 $\pm\,0.0$	92.6 $\pm\,0.2$	2.5 $\pm\,0.2$	4.9 $\pm\,0.0$	89.9 $\pm\,0.5$	4.5 $\pm\,0.3$
Reduction	Beta Mean	88.2	81.4	96.7 $\pm 0.1$	90.3 $\pm 1.0$	3.3 $\pm 1.0$	6.4 $\pm 0.1$	89.2 $\pm 0.7$	3.9 $\pm 0.6$
	Median	88.1	81.2	96.8 $\pm 0.5$	90.1 $\pm 1.5$	3.6 $\pm 0.6$	6.3 $\pm 0.9$	88.6 $\pm 0.2$	4.8 $\pm 0.6$
	Max	88.1	79.0	98.1 $\pm 0.0$	95.1 $\pm 0.2$	1.2 $\pm 0.1$	3.8 $\pm 0.1$	85.9 $\pm 0.2$	6.8 $\pm 0.4$
	Min	87.7	80.9	95.5 $\pm 0.3$	82.5 $\pm 0.7$	9.0 $\pm 1.3$	8.6 $\pm 0.6$	88.4 $\pm 0.9$	4.1 $\pm 0.3$

Table 5: Unified ablation results for StreamGuard with the Llama-3-8B backbone. Reduction , Rollout Mix , and Sup Density vary the rollout-score aggregation rule, rollout-generator composition, and dense-prefix / strided-tail supervision schedule, respectively; Reference is the main configuration used in the paper. Prompt Mod and Resp Mod report macro-averaged F1 over the standard benchmark suites; Appendix C provides the dataset-level moderation breakdowns. QwenGuardTest and XSTest-Resp use the same streaming and overblocking protocols as in § 3.1 . We report mean

\pm

std over three seeds where shown.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Streaming	ToxiC	OAIMod	Aegis	Aegis2	SSTest	HarmB	WildG	Avg
LlamaGuard3-8B	no	53.8	79.5	71.5	76.4	99.5	99.0	76.4	79.4
LlamaGuard4-12B	no	51.3	73.5	67.8	70.6	98.0	97.2	73.0	75.9
WildGuard-7B	no	70.8	72.1	89.4	80.7	99.5	98.9	88.9	85.8
ShieldGemma-9B ( Zeng et al., 2024 )	no	69.4	82.1	70.3	72.5	83.7	60.6	54.2	70.4
ShieldGemma-27B	no	72.9	80.5	69.0	71.6	84.4	57.3	54.3	70.0
NemoGuard-8B	no	75.6	81.0	81.4	86.8	98.5	75.2	81.6	82.9

Appendix

Table 6: Input-moderation F1; Avg denotes the macro-average. For prior systems, we report published results. We report mean

\pm

std over three seeds for StreamGuard models. The Streaming column indicates whether a model supports token-level streaming moderation.

Model	Streaming	HarmB	SRLHF	BeaverTails	XSTestR	Aegis2	WildG	Avg
LlamaGuard3-8B	no	84.5	45.2	67.9	89.8	66.1	69.5	70.5
LlamaGuard4-12B	no	83.3	42.5	68.6	88.9	63.7	66.4	68.9
WildGuard-7B	no	86.3	64.2	84.4	94.7	83.2	75.4	81.4
ShieldGemma-9B	no	60.4	44.2	62.4	86.3	70.8	49.9	62.3
ShieldGemma-27B	no	62.9	52.6	67.6	83.0	74.9	52.4	65.6
NemoGuard-8B	no	81.4	57.6	78.5	86.2	87.6	77.5	78.1

Appendix

Table 7: Output-moderation F1; Avg denotes the macro-average. For prior systems, we report published results. StreamGuard models are evaluated in our simulated streaming protocol, processing responses one token at a time and counting a response as unsafe if it triggers under the debounced rule in § 3.1 . We report mean

\pm

std over three seeds for StreamGuard models. The Streaming column indicates whether a model supports token-level moderation during generation.

Factor	Variant	ToxiC	OAIMod	Aegis	Aegis2	SSTest	HarmB	WildG	Avg
Reduction	Mean	77.4 $\pm\,0.5$	75.0 $\pm\,0.4$	88.5 $\pm\,0.2$	87.9 $\pm\,0.1$	99.5 $\pm\,0.0$	99.7 $\pm\,0.1$	89.5 $\pm\,0.2$	88.2
Reduction	Beta Mean	77.0 $\pm\,0.9$	75.0 $\pm\,2.5$	88.4 $\pm\,0.3$	87.8 $\pm\,0.5$	99.5 $\pm\,0.0$	99.9 $\pm\,0.1$	89.5 $\pm\,0.0$	88.2
Reduction	Median	77.0 $\pm\,2.0$	73.3 $\pm\,1.3$	89.9 $\pm\,0.2$	87.7 $\pm\,0.2$	99.5 $\pm\,0.0$	99.9 $\pm\,0.1$	89.6 $\pm\,0.3$	88.1
Reduction	Max	77.4 $\pm\,0.7$	73.6 $\pm\,0.2$	89.2 $\pm\,0.5$	87.9 $\pm\,0.5$	99.5 $\pm\,0.0$	99.9 $\pm\,0.1$	89.5 $\pm\,0.3$	88.1
Reduction	Min	75.9 $\pm\,1.7$	73.0 $\pm\,0.9$	88.4 $\pm\,0.7$	87.7 $\pm\,0.1$	99.5 $\pm\,0.0$	99.9 $\pm\,0.1$	89.5 $\pm\,0.3$	87.7
Rollout Mix	Four-model mixture (16)	77.4 $\pm\,0.5$	75.0 $\pm\,0.4$	88.5 $\pm\,0.2$	87.9 $\pm\,0.1$	99.5 $\pm\,0.0$	99.7 $\pm\,0.1$	89.5 $\pm\,0.2$	88.2

Appendix

Table 8: Dataset-level input-moderation breakdown for the ablations in Table 5 . This appendix table decomposes the aggregate input-moderation result from the main text into per-benchmark F1 scores for the same Llama3-StreamGuard-8B variants, varying the reduction rule used to aggregate rollout oracle scores, the rollout-generator mix across the compared single-model, two-model, and four-model settings, and the supervision-density schedule. Consistent with the paper’s main ablation story, input-side moderation remains tightly clustered across these practical target-construction choices, indicating that forecasting-based supervision is robust rather than dependent on a brittle implementation detail. Avg denotes the macro-average across benchmarks; supervision-schedule notation follows Table 5 ; reported values are mean

\pm

standard deviation over three seeds where shown.

Factor	Variant	HarmB	SRLHF	BeaverTails	XSTestR	Aegis2	WildG	Avg
Reduction	Mean	83.3 $\pm\,0.3$	69.2 $\pm\,0.3$	86.6 $\pm\,0.2$	89.9 $\pm\,0.5$	84.5 $\pm\,0.1$	77.8 $\pm\,0.1$	81.9
Reduction	Beta Mean	83.4 $\pm 0.7$	67.6 $\pm 0.5$	86.5 $\pm 0.2$	89.2 $\pm 0.7$	84.7 $\pm 0.8$	77.3 $\pm 1.1$	81.4
Reduction	Median	83.5 $\pm 0.6$	67.3 $\pm 1.0$	86.1 $\pm 0.0$	88.6 $\pm 0.2$	84.3 $\pm 0.9$	77.2 $\pm 0.5$	81.2
Reduction	Max	79.8 $\pm 0.2$	69.8 $\pm 0.8$	85.9 $\pm 0.2$	85.9 $\pm 0.2$	81.1 $\pm 0.5$	71.2 $\pm 1.3$	79.0
Reduction	Min	84.7 $\pm 0.1$	63.3 $\pm 0.7$	85.6 $\pm 0.2$	88.4 $\pm 0.9$	85.0 $\pm 0.6$	78.6 $\pm 0.8$	80.9
Rollout Mix	Four-model mixture (16)	83.3 $\pm\,0.3$	69.2 $\pm\,0.3$	86.6 $\pm\,0.2$	89.9 $\pm\,0.5$	84.5 $\pm\,0.1$	77.8 $\pm\,0.1$	81.9

Appendix

Table 9: Dataset-level output-moderation breakdown for the ablations in Table 5 . This appendix table decomposes the aggregate output-moderation result from the main text into per-benchmark F1 scores under the same streaming evaluation protocol used throughout the paper, where a response is marked unsafe if the guardrail triggers at any point during generation. Read together with the main ablation table’s streaming-timing and overblocking columns, these results support the central ablation story: StreamGuard remains competitive across a broad range of target-construction variants, while the meaningful differences come from the operating-point trade-off between more aggressive early intervention and better benign calibration. Avg denotes the macro-average across benchmarks; supervision-schedule notation follows Table 5 ; reported values are mean

\pm

standard deviation over three seeds where shown.

model	Cross-Tokenizer	ToxiC	OAIMod	Aegis	Aegis2	SSTest	HarmB	WildG	Avg
Qwen3Guard-Stream-0.6B-strict	No	72.0	68.3	85.2	84.9	98.0	97.2	87.1	84.7
Qwen3Guard-Stream-0.6B-loose	No	75.5	76.0	77.7	81.7	96.9	96.8	86.0	84.4
Llama3-StreamGuard-1B (ours)	No	74.9 $\pm\,0.3$	71.5 $\pm\,0.2$	89.0 $\pm\,0.3$	87.7 $\pm\,0.2$	98.6 $\pm\,0.2$	94.8 $\pm\,2.9$	88.6 $\pm\,0.2$	86.4
Gemma3-StreamGuard-0.3B (ours)	Yes	60.1 $\pm 3.0$	63.8 $\pm 0.6$	83.9 $\pm 0.5$	82.9 $\pm 0.5$	96.0 $\pm 0.6$	89.1 $\pm 4.5$	83.7 $\pm 0.7$	79.9
Gemma3-StreamGuard-1B (ours)	Yes	74.0 $\pm 0.4$	71.2 $\pm 0.4$	87.4 $\pm 1.3$	86.6 $\pm 0.2$	98.5 $\pm 0.0$	89.5 $\pm 2.8$	88.4 $\pm 0.2$	85.1
Qwen3-StreamGuard-0.6B (ours)	Yes	66.9 $\pm 0.8$	67.7 $\pm 0.8$	86.0 $\pm 0.3$	85.1 $\pm 0.3$	96.9 $\pm 0.5$	88.1 $\pm 0.9$	85.8 $\pm 0.4$	82.3

Appendix

Table 10: Dataset-level input-moderation results for the cross-tokenizer transfer setting, complementing Table 4 . “Cross-Tokenizer” indicates whether prefix-level future-risk supervision is transferred from a source model with a different tokenizer than the target model. Despite the tokenizer mismatch, transferred StreamGuard models remain competitive on standard input-moderation benchmarks. Avg denotes the macro-average across datasets; reported values for StreamGuard models are mean

\pm

standard deviation over three seeds where shown.

model	Cross-Tokenizer	HarmB	SRLHF	BeaverTails	XSTestR	Aegis2	WildG	Avg
Qwen3Guard-Stream-0.6B-strict	No	83.1	62.8	84.5	84.8	81.4	76.3	78.8
Qwen3Guard-Stream-0.6B-loose	No	80.6	61.7	84.0	83.3	81.4	75.8	77.8
Llama3-StreamGuard-1B (ours)	No	82.4 $\pm\,0.2$	68.0 $\pm\,0.7$	86.2 $\pm\,0.1$	84.6 $\pm\,0.7$	82.9 $\pm\,0.3$	77.2 $\pm\,0.3$	80.2
Gemma3-StreamGuard-0.3B ( ours )	Yes	77.2 $\pm 2.8$	65.6 $\pm 0.4$	85.2 $\pm 0.4$	76.1 $\pm 2.4$	80.7 $\pm 0.6$	67.1 $\pm 3.4$	75.3
Gemma3-StreamGuard-1B ( ours )	Yes	82.7 $\pm 0.8$	69.7 $\pm 0.4$	87.2 $\pm 0.2$	87.2 $\pm 0.0$	84.7 $\pm 0.3$	76.3 $\pm 1.0$	81.3
Qwen3-StreamGuard-0.6B ( ours )	Yes	80.9 $\pm 0.9$	68.8 $\pm 0.4$	86.3 $\pm 0.4$	82.8 $\pm 0.8$	83.2 $\pm 0.7$	74.4 $\pm 0.8$	79.4

Appendix

Table 11: Dataset-level output-moderation results for the cross-tokenizer transfer setting, complementing Table 4 . Models are evaluated under the same streaming output-moderation protocol used in the main paper, where a response is marked unsafe if it triggers at any point during generation. Cross-tokenizer transfer remains effective across target families, with transferred StreamGuard models preserving strong output-moderation performance. Avg denotes the macro-average across datasets; reported values for StreamGuard models are mean

\pm

standard deviation over three seeds where shown.

Model	Role	Latency (ms)	Decision Ratio		Decision/s
Model	Role	Latency (ms)	8B	70B	Decision/s
Llama3-8B-Instruct	Generator	9.8	-	-	-
Llama3-70B-Instruct	Generator	99.2	-	-	-
Gemma3-StreamGuard-0.3B	Guardrail	2.4	0.243	0.024	420
Gemma3-StreamGuard-1B	Guardrail	3.7	0.375	0.037	271
Qwen3-StreamGuard-0.6B	Guardrail	4.3	0.439	0.043	232

Appendix

Table 12: Latency measurements in the steady-state decoding regime. We measure from a prefix length of 1024 tokens and average over the next 1024 decoding steps, using 100 runs per setup. All experiments use NVIDIA H100 GPUs with HuggingFace and StaticCache ; Llama3-70B-Instruct is measured on two GPUs. Decision ratios normalize guardrail latency by the average per-token latency of the Llama3-8B-Instruct and Llama3-70B-Instruct generators and are the primary quantity of interest for pipelined moderation.

LLM Guardrails

Large language model guardrails detect or constrain harmful outputs, prompt attacks, and unsafe agent actions before they cause harm.

See all 185 papers

SentGuard: Sentence-Level Streaming Guardrails for Large Language Models
Jun 1, 2026Jiaqi Yu, Xin Wang, Yixu Wang +4Language Model Safety Evaluation LLM Guardrails
1Fudan University · 2Shanghai AI Laboratory
Stop Early, Spend Less: Hidden-State Probes as a Practical Recipe for Streaming Moderation of LLM Outputs
Jun 9, 2026Huizhen Shu, Xuying Li, Piao XueLLM Guardrails Safety Filtering
ModelOneAI · yunshanai
Safety Reconstructed: Generative Modeling via Masked Diffusion Builds Strong Safety Guardrails
Sep 27, 2026Gert Lek, Abele Malan, Chaoyi Zhu +3LLM Safety Benchmarks LLM Guardrails
University of Neuchâtel · Delft University of Technology · IBM Research +1

Jun 1, 2026J/K move · Enter open · S save

SentGuard: Sentence-Level Streaming Guardrails for Large Language Models

Jiaqi Yu, Xin Wang, Yixu Wang, Jie Li+3

1Fudan University · 2Shanghai AI Laboratory

Language Model Safety Evaluation LLM Guardrails Content Moderation Safety Filtering LLM Safety LLM Safety Monitoring

Open paper AskPDF arXiv

Large language models increasingly stream long, reasoning-intensive responses in real time, making when to moderate as critical as whether to moderate. Existing guardrails fall into two unsatisfactory extremes: response-level methods delay intervention until the full output is generated, whereas token-level methods act on incomplete semantics, often producing unstable decisions and excessive guard invocations. To address this challenge, we propose SentGuard, a sentence-level streaming guardrail that operates in parallel with generation. A lightweight waiting buffer groups streamed tokens into sentence chunks and releases only verified chunks to the user, introducing a small offset that enables SentGuard to assess the current prefix while the target LLM decodes subsequent content. To support this, we construct StreamSafe, a benchmark with structured per-sentence annotations across 8 harm categories, capturing the evolution of safety risks across both reasoning and response segments. We further train SentGuard with a coarse-to-fine objective to detect unsafe intent as soon as it emerges at sentence boundaries. Experiments on 5 safety benchmarks show that SentGuard outperforms existing baselines, detecting 90.5% of unsafe cases within two sentences while maintaining a low streaming false-positive rate of 7.41%.

Abstract

LLM Guardrails

Explore similar work

SentGuard: Sentence-Level Streaming Guardrails for Large Language Models

Stop Early, Spend Less: Hidden-State Probes as a Practical Recipe for Streaming Moderation of LLM Outputs

Safety Reconstructed: Generative Modeling via Masked Diffusion Builds Strong Safety Guardrails