RASSG: Repair-Aware Semantic Survival Gating for Streaming Tool Execution

Daijin Hu; Xi Wang

cs.AIOct 10, 2026

RASSG: Repair-Aware Semantic Survival Gating for Streaming Tool Execution

Authors: Daijin Hu, Xi Wang

Organizations: Beihang University

Abstract

Streaming speech assistants can launch tools before users finish speaking, overlapping execution with incoming speech to reduce user waiting time. However, subsequent self-corrections can invalidate apparently stable arguments, causing stale execution whose irreversible effects persist even after the final call is corrected. We propose Repair-Aware Semantic Survival Gating (RASSG) to jointly control tool launch and cancellation during streaming speech. RASSG launches a tool when three consecutive speech recognition results arrive with identical nonempty arguments, then combines calibrated argument invalidity risk with execution costs to decide cancellation at the last feasible opportunity before effects become irreversible. Evaluation on ReCAST, FDB-v3, and Audio2Tool uses a controlled execution clock with fixed tool runtimes and cancellation parameters. On ReCAST's prospective-repair subset, RASSG reduces the stale execution rate from LocalAgreement-2's 50.00% to 3.15%, with a mean delay of 3.676s across successful trajectories. The stale reduction also holds against every early-execution baseline on both public speech datasets. Ablations with identical initial launches support active cancellation, showing the value of controlling ongoing calls before their effects become irreversible. Code and data are available at https://github.com/daijinhu/RASSG.

Figures & tables

Figure 1: An illustrative scheduling request changes from 3 pm to 4 pm. Confidence-based early commitment leaves the obsolete call running past its irreversible deadline (the action boundary), while conservative delayed commitment waits for the correction before launching. RASSG launches on stable arguments, monitors invalidity risk during execution, and cancels the old call before the deadline, then launches the revised request.

Figure 2: RASSG control loop. Agreement across three actual ASR arrivals triggers launch. New arguments update active execution, while calibrated invalidity risk, tool configuration, and execution state determine cancellation at the last feasible opportunity.

	ReCAST		FDB-v3		Audio2Tool
Method	Stale execution rate (%) $\downarrow$	Successful delay (s) $\downarrow$	Stale execution rate (%) $\downarrow$	Successful delay (s) $\downarrow$	Stale execution rate (%) $\downarrow$	Successful delay (s) $\downarrow$
EOU	0.58	8.676	40.00	23.880	52.73	9.196
LocalAgreement-2	50.00	3.954	65.00	9.210	62.91	8.135
Confidence commit	21.08	5.633	73.00	9.507	73.92	7.910
Stream RAG	50.00	5.312	86.00	19.997	84.50	8.255
Speculative Tool Calling	50.00	3.480	77.00	9.280	81.05	8.216

Table 1: Comparison of RASSG and baseline execution policies on ReCAST, FDB-v3, and Audio2Tool. Lower is better for all metrics.

Component removals
Variant	Stale execution rate (%) $\downarrow$	Successful delay (s) $\downarrow$
RASSG (full)	3.15	3.676
Remove active cancellation	49.92	4.434
Remove progress reuse	3.06	4.510
Launch consensus
One-arrival consensus	28.78	3.038

Table 2: Component ablations of RASSG on ReCAST, changing one design choice at a time.

Tool Use

Tool use studies how AI agents select, invoke, and combine external tools to solve tasks beyond their internal knowledge.

See all 91 papers

When Does Streaming Tool Use Help? Characterizing Tool-Intent Stabilization in Streaming Retrieval-Augmented Generation
Jun 18, 2026Elroy GalbraithRetrieval-Augmented Generation Information Retrieval
SMG Labs
Beyond Fresh Starts: Stateful Inference for Streaming ASR in Conversational Voice Agents
Aug 22, 2026Sameep Chattopadhyay, Alexander Erdmann, Mari OstendorfConversational Speech Recognition Streaming ASR
University of Washington · SRI International
Hiding Tool Latency in On-Device Cascaded Voice Agent through Speculative Execution
Oct 6, 2026Kyudan Jung, Hyunsin Park, Yoonhyung Lee +5Conversational Agents
Qualcomm AI Research · KAIST AI

Jun 18, 2026J/K move · Enter open · S save

When Does Streaming Tool Use Help? Characterizing Tool-Intent Stabilization in Streaming Retrieval-Augmented Generation

Elroy Galbraith

SMG Labs

Retrieval-Augmented Generation Information Retrieval RAG Evaluation

Open paper AskPDF arXiv

Streaming Retrieval-Augmented Generation (Streaming RAG) hides tool latency by issuing retrieval queries in parallel with the user's still-arriving input, before the utterance is complete. Speculation can only help, though, when the correct query becomes determinable before the user stops speaking or typing -- a property of the query, not the system. We name and measure this property, tool-intent stabilization: the point in the input stream at which a speculative query's retrieval converges on the answer-bearing result. On the CRAG benchmark (1371 validation questions) we (i) characterize how stabilization is distributed across queries; (ii) derive a model-agnostic bound H on the share of tool latency hideable behind the remaining input, given tool latency L and input cadence delta; (iii) validate it against a working streaming pipeline; and (iv) ask which query properties predict early versus late stabilization. Stabilization is typically early: at a realistic operating point a 73.9% streamable fraction of the benchmark admits latency hiding, and H acts as a conservative aggregate floor that realized savings meet or exceed -- though it does not predict savings query by query. Question type yields a statistically significant but small early/late split. The study needs no model training and runs on commodity CPU hardware; a dense-retriever replication confirms the early-stabilization effect is not a BM25 lexical artifact.

Abstract

Tool Use

Explore similar work

When Does Streaming Tool Use Help? Characterizing Tool-Intent Stabilization in Streaming Retrieval-Augmented Generation

Beyond Fresh Starts: Stateful Inference for Streaming ASR in Conversational Voice Agents

Hiding Tool Latency in On-Device Cascaded Voice Agent through Speculative Execution