ActiveEvidence: Decision-Focused Evidence Graphs with Verifiable Acquisition Gaps

Miaobo Hu; Shuhao Hu; Xiaobo Guo; Xin Wang; Bokun Wang; Peng Zhang; Daren Zha

cs.AIOct 14, 2026

ActiveEvidence: Decision-Focused Evidence Graphs with Verifiable Acquisition Gaps

Authors: Miaobo Hu, Shuhao Hu, Xiaobo Guo, Xin Wang, Bokun Wang, Peng Zhang, Daren Zha

Organizations: School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China

Abstract

A multimodal agent must decide whether visual evidence is worth acquiring, which candidate or resolution to query, and when further interaction can no longer change its decision. We study this problem under a strict free-metadata/paid-observation boundary, stochastic vector costs, and a frozen execution trace, and treat it as decision-certifiable active evidence acquisition. A Counterfactual Evidence Interaction Graph (CEIG) represents unary evidence value, signed second-order interaction, and directed conditional harm while maintaining explicit score enclosures for unresolved interactions; a decision-weighted enclosure objective tightens the bounds that can change action identity. Gap-directed refinement evaluates only interactions that can displace the incumbent acquisition and stops once an $ε$ -decision certificate shows that unresolved terms cannot change the selected action beyond a declared tolerance. The certificate is model-relative: it verifies policy-consistent optimality under the pinned learned score, and semantic quality is measured separately through post-freeze labels. Paired continuation rollouts learn the downstream value of paid previews, an online operational cost filter is separated from independent finite-policy-family trajectory certification, and an acquisition monitor with an independently implemented verifier binds every certificate to the frozen trace. On 640 cases split into 384/128/128 training, validation, and test cases, the controller reaches task score 0.842, grounding F1 0.816, evidence recall 0.873, 1.74 accepted calls, 286.5K visual tokens, and 742ms p95 latency. Dense replay of issued certificates shows 0.992 action agreement and a 0.988 certificate rate while evaluating 19.6 instead of 191.4 expensive interaction terms per decision.

Figures & tables

method	unit	pair.	budget/risk	cert.	oracle	A/R	replay	audit	semantic
ActiveEvidence	whole image	yes	call cap / traj. risk	yes	post-freeze	yes	cross-impl.	0.973	held-out quality
VisionThink	whole image	N/R	N/R	N/R	not explicit	partial	N/R	N/R	task benchmarks
AdaptVision	crop/resolution	N/R	N/R	N/R	not explicit	partial	N/R	N/R	task benchmarks
AdaTooler-V	visual tools	N/R	N/R	N/R	not explicit	partial	N/R	N/R	task benchmarks
DART	visual tools	N/R	N/R	N/R	not explicit	partial	N/R	N/R	task benchmarks
ToolScope	multi-tool	N/R	N/R	N/R	not explicit	partial	N/R	N/R	task benchmarks

Table 1. Contract fields reported by the closest adaptive visual-acquisition methods: pairwise evidence interactions (pair.), call cap and trajectory-risk statement (budget/risk), decision-score certificate (cert.), and independently checked attack result (audit); “N/R” marks a field the source leaves unreported.

Figure 1. Overview of ActiveEvidence. State inputs feed learned-value prediction (need and preview-value heads and the CEIG). Decision-focused computation filters infeasible actions, keeps the incumbent

S^{*}=\widehat{S}_{t}(b_{t})

and challenger bounds

[L,U]=[\ell_{t},u_{t}]

, and resolves the widest unresolved edge of the top challenger until

\max U-S^{*}\leq\epsilon

certifies the action. Previews and acquisitions re-score the state, STOP and ABSTAIN terminate, and all events are frozen and independently verified before evaluation. A left-to-right pipeline with five panels above a verification band. State inputs (request, free metadata such as ID, type, and dimension, paid evidence that contains pixels, and the budget) enter learned-value prediction, where offline supervision trains the need and preview-value heads and a CEIG graph over candidates A to F has positive, negative, and harm edges. Decision-focused computation applies a cost filter to obtain feasible actions, keeps an incumbent score S-star and challenger bounds L to U, selects the challenger with the largest U, resolves its widest unresolved edge, and repeats. The epsilon-decision certificate checks that max U minus S-star is at most epsilon. Actions are PREVIEW and ACQUIRE, which loop back to update and re-score the state, and the terminal STOP and ABSTAIN. The verification band passes attempted, rejected, and accepted events through trace freeze, an independent verifier, and evaluation.

method	task	ground.	evidence	coverage	calls	V tok.	p95 ms	viol.
Text-only	0.687	0.421	0.000	0.844	0.00	0.0K	218	0.000
Always-on	0.817	0.789	0.948	0.969	4.93	808.6K	1687	0.126
Random	0.731	0.596	0.623	0.875	2.91	476.8K	1034	0.071
Static metadata	0.764	0.674	0.704	0.891	2.43	398.7K	889	0.052
Adaptive vision	0.806	0.758	0.818	0.914	2.32	381.4K	917	0.047
Weighted metadata	0.782	0.711	0.749	0.898	2.24	367.9K	846	0.043

Table 2. Held-out semantic quality and resource use on the 128-case test split under the shared Qwen2.5-VL-7B-Instruct decoder, preview permission, action cap, vector budget, and three controller seeds. Coverage is the answered-case fraction; higher is better for task, grounding, evidence recall, and coverage, lower for calls, visual tokens, p95 latency, and violations. Bold marks the best and underline the second-best value among evidence-acquiring policies; text-only is the no-acquisition reference.

Figure 2. Task score against visual tokens per case (K) for Table 2 : (a) all methods and (b) acquisition methods enlarged. Green diamonds mark matched prior mechanisms; the orange star marks ActiveEvidence. Two task-score versus token plots show all eighteen comparison methods and a zoom of the acquisition methods. Matched baselines are green diamonds and ActiveEvidence is an orange star.

Appendix figures & tables29 assets

Supplementary material from the paper’s appendix.

Appendix

policy	activ.	attempt.	accept.	V tok.	lat. (ms)	ID hit	exhaust.
text only	0.000	0.00	0.00	0.00	1.00	0.000	0.000
always-on	1.000	2.24	1.63	13.00	4.25	0.500	0.625
static selector	1.000	2.01	1.50	13.00	4.25	0.625	0.500
active evidence	0.625	1.42	1.03	10.00	3.50	0.500	0.375

Appendix

Table 3. Mechanism-level and row-audit quantities under the metadata-only contract. Attempted and accepted calls are separate ledger quantities; ID hit and exhaustion are post-freeze diagnostics, and the aggregate call means come from the frozen row-level audit. Bold marks the best value among the three acquiring policies (lower is better for activation, attempted and accepted calls, tokens, latency, and exhaustion; higher for ID hit); text only is the no-acquisition reference.

study	evidence type	retained result and boundary
Mechanism screen	metadata fixture (proxy)	Eight metadata-only records: active selection activates 0.625 of rows, uses selected-image count 1.250, and reaches image-ID proxy 0.500. The screen is metadata-only: image decoding and vision-model execution are outside this fixture; these are mechanism diagnostics.
Static control	metadata fixture (proxy)	The static selector activates every row, uses 1.625 metadata-ranked images, and reaches image-ID proxy 0.625. This is a composite mechanism control; the held-out semantic task score is 0.764.
Field provenance	provenance record	TextConf-v0.3, VisualQuality-v0.4, and TextImageRel-v0.5 bind visible inputs and access records; ECE for $c_{i},q_{ij},r_{ij}$ is 0.041/0.036/0.039. Phase-specific oracle-access counts are reported in Appendix D.6 .
Matched policy controls	held-out test	The matched surface defines task/grounding/calls/V-tokens/p95 for the controls in Table 2 ; all control rows are joined under the same held-out receipt.
Held-out visual quality and cost	held-out test	The pinned Qwen2.5-VL-7B-Instruct study covers the 128-case test split and three controller seeds; the active semantic row is 0.842/0.816/0.873 with 1.74 calls, 286.5K visual tokens, and 742ms p95 latency.
Audit injection and robustness	held-out test	Injected trace-violation detection is 0.973 with clean-trace FPR 0.012; post-execution mutation detection ranges from 0.944–0.992, and matched transfer, hard-negative, and conflict task-score drops are 0.021, 0.028, and 0.034.

Appendix

Table 4. Experiment ledger and evidence boundary. The evidence type separates metadata-fixture proxies, provenance records, and held-out test results.

Evidence receipt	Supported interpretation
Manifest SHA-256 and 12480/12471 image hashes	Asset identity and byte-level matching
Accepted-call means 1.63/1.50/1.03 and exhaustion 0.625/0.500/0.375	Final-checkpoint vector-budget accounting
Field ECE 0.041/0.036/0.039	Calibrated selection-visible provenance
Exact/action/metric replay 0.985/0.972/0.957	Cross-implementation trace equivalence
Injected violations detected at 0.973	Audit-stress detection under the declared threat model
Eight-row fixture and held-out semantic table	Mechanism unit testing is separated from task quality

Appendix

Table 5. Evidence receipts and their supported interpretation.

stress	protocol	task	V tok.	audit outcome
candidate-pool size	small/medium/large fixed pools	0.833/0.842/ 0.846	231.7 /286.5/338.2K	0.969/0.973/0.971 detection
hard negatives	near-duplicate and distractor candidates	0.814	302.8K	0.976 detection; 0.028 task drop
conflicting evidence	contradictory captions or regions	0.808	310.6K	0.979 detection; 0.034 task drop
metadata corruption	hash, size, dimension, and score perturbations	0.832	293.4K	0.981 detection
budget shifts	call/token/latency vectors	0.823 (worst shift)	272.9K	0.029 violation; 0.972 detection
injected trace violations	oracle access, duplicate IDs, post-freeze edits	0.841	287.2K	0.973 detection; 0.012 clean FPR

Appendix

Table 6. Robustness and trace-audit stress matrix on the 128-case held-out split. Task is higher-is-better and visual tokens (V tok.) lower-is-better; detection is the access-injection detection rate under its declared denominator. Bold marks the best value across these stress settings.

policy	attempted	accepted	final $B_{\rm call}$	$E_{i}$
always-on	2.24	1.63	0.37	0.625
static selector	2.01	1.50	0.50	0.500
active evidence	1.42	1.03	0.97	0.375

Appendix

Table 7. Oracle-free row-level budget audit. Attempted calls and final remaining call budgets are reported from the row-level receipt; accepted calls and exhaustion use the frozen held-out audit. Bold marks the best value per column (lower is better for calls and exhaustion, higher for the final call budget).

gate	ranker	stopper	activ. $\downarrow$	selected $\downarrow$	ID proxy $\uparrow$	quality $\uparrow$
off	static	fixed	1.000	1.625	0.625	0.776
on	static	fixed	0.625	1.250	0.625	0.792
off	active	fixed	1.000	1.625	0.500	0.817
on	active	fixed	0.625	1.250	0.500	0.829
off/on	random	fixed	0.750	1.375	0.375	0.731
on	learned	learned	0.625	1.125	0.625	0.842

Appendix

Table 8. Predeclared mechanism controls for gate and ranker attribution on the eight-record fixture. Activation and selected candidates are lower-is-better under the mechanism contract; ID proxy and held-out quality are higher-is-better. Bold marks the best value per column.

policy	p95 (ms)	VRAM (GB)	abstain	fail.	disp.
text only	218	11.4	0.156	0.042	0.006
always on	1687	18.7	0.031	0.051	0.008
static selector	889	15.1	0.102	0.036	0.007
random selector	1034	15.6	0.117	0.049	0.009
active evidence	742	15.8	0.078	0.023	0.005

Appendix

Table 9. Held-out operational latency, memory, abstention, and failure outcomes under the same pinned contract. Bold marks the best and underline the second-best value among evidence-acquiring policies for p95 latency, VRAM, failures, and dispersion; text-only is the no-acquisition reference, and abstention is a selective-prediction rate without a preferred direction.

model	int. MAE $\downarrow$	sign AUC $\uparrow$	comp. $\uparrow$	redund. $\downarrow$	conflict $\downarrow$
Unary only	0.184	0.611	0.642	0.176	0.139
Latent pairwise	0.119	0.786	0.747	0.132	0.094
Bundle regression	0.142	0.738	0.709	0.147	0.112
No re-scoring	0.083	0.887	0.822	0.104	0.063
No uncertainty calibration	0.077	0.901	0.839	0.096	0.058
Full CEIG	0.071	0.914	0.861	0.081	0.047

Appendix

Table 10. Direct attribution of explicit interaction supervision on the frozen test split. Arrows indicate the preferred direction. Bold marks the best and underline the second-best value per column.

objective	task $\uparrow$	int. MAE $\downarrow$	dense agr. $\uparrow$	cert. $\uparrow$	edges $\downarrow$	regret $\downarrow$
No tightness	0.838	0.069	0.964	0.923	34.2	0.012
Uniform tightness	0.840	0.073	0.978	0.956	26.8	0.007
Decision-weighted (default)	0.842	0.071	0.992	0.988	19.6	0.003

Appendix

Table 11. Attribution of certificate-oriented CEIG training on the frozen test split. Arrows give the preferred direction; the three rows differ only in the training objective. Bold marks the best and underline the second-best value per column.

controller	nom. $\downarrow$	+20% $\downarrow$	+50% $\downarrow$	mixed $\downarrow$	task $\uparrow$	CVaR $\downarrow$
Nominal hard filter	0.061	0.148	0.319	0.274	0.788	0.151
Nominal filter + dual price	0.045	0.109	0.238	0.194	0.801	0.108
Conformal feasible set	0.025	0.052	0.113	0.079	0.814	0.058
Full validation-calibrated controller	0.018	0.024	0.041	0.029	0.823	0.032

Appendix

Table 12. Budget safety under execution-cost shifts. Violation is the fraction of test trajectories with any negative budget component; CVaR uses the declared cost-overrun loss. Bold marks the best and underline the second-best value per column.

Figure 3. Budget safety under execution-cost shifts for the four controllers of Table 12 : (a) violation rate per shift condition and (b) worst-shift task score against cost CVaR. A heatmap gives violation rates for four controllers under four cost conditions, followed by a scatter of worst-shift task score against cost CVaR. Full-controller heatmap values are bold.

policy	$n_{\rm cert}$	$v_{\theta}$	$U_{\theta}$ $\downarrow$	approved	confirm. viol. $\downarrow$
Point-cost filter	500	31	0.082	no	0.061
Adaptive upper-cost filter	500	13	0.041	yes	0.025
Complete-policy certification	500	8	0.028	yes	0.018

Appendix

Table 13. Trajectory-risk certification for the frozen policy family. The certification bound covers the complete adaptive episode, including previews, refinement, failed dispatches, and terminal generation. Bold marks the best and underline the second-best value for the bound and the confirmation rate.

attack	cases	detect. $\uparrow$	false acc. $\downarrow$	impact $\downarrow$
Oracle field before freeze	200	0.992	0.004	0.061
Candidate ID or hash substitution	200	0.986	0.006	0.047
Rejected attempt removed	200	0.971	0.010	0.013
Cost under-reporting	200	0.958	0.016	0.028
Feasible action omitted	200	0.962	0.014	0.036
Action score modified	200	0.968	0.010	0.043

Appendix

Table 14. Decision and trace certificate stress test. Every attack category has an explicit denominator, detection rate, adversarial false-acceptance rate, and metric-impact field; clean-trace FPR is stored in the receipt. Bold marks the best and underline the second-best value per column over the ten categories.

Figure 4. Certificate attacks of Table 14 : (a) detection rate and (b) adversarial false-acceptance rate and metric impact per category. Dashed lines mark the overall detection rate (0.967) and overall metric impact (0.042). Two aligned dot plots list ten attack categories. Detection rates lie between 0.944 and 0.992. False-acceptance rates lie between 0.004 and 0.020, and metric impacts between 0.013 and 0.068.

attack	cases	reject. $\uparrow$	wrong $\downarrow$	unres. $\downarrow$	chg. $\downarrow$
Evaluation-label access before freeze	200	198	1	1	1
Undeclared pixel preview	200	195	3	2	2
Forged cost plus action-score edit	200	192	5	3	3
Omitted feasible action plus ledger edit	200	191	6	3	4
Post-freeze chain reconstruction	200	189	7	4	5

Appendix

Table 15. Access-boundary and coordinated certificate attacks. Rejected, incorrectly accepted (wrong), and unresolved outcomes sum to the attack denominator; chg. counts incorrectly accepted traces that change a reported metric. Trusted monitor or commitment-store compromise is a separately declared threat-model condition. Bold marks the best and underline the second-best value per column.

rule	agr. $\uparrow$	regret $\downarrow$	cert. $\uparrow$	edges $\downarrow$	ctrl. ms $\downarrow$	ver. ms $\downarrow$
Dense evaluation	1.000	0.000	1.000	191.4	184.6	38.2
Fixed- $L$ neighbors	0.938	0.024	0.914	48.0	56.3	14.7
Random	0.912	0.038	0.875	32.5	49.1	12.5
Widest envelope	0.969	0.011	0.953	28.7	46.8	11.9
Top challenger (default)	0.992	0.003	0.988	19.6	41.8	9.4
Per-compute priority	0.994	0.002	0.991	16.8	38.5	8.7

Appendix

Table 16. Gap-directed refinement rules with identical model weights. The local regret is relative to the pinned dense controller; expensive-edge counts and latencies are measured on the same held-out cases. Dense evaluation is the reference row; bold marks the best and underline the second-best value among the sparse rules.

Figure 5. Refinement rules of Table 16 : (a) dense agreement and certificate rate, (b) expensive edges, and (c) controller and independent-verifier time. Per-compute priority retains high consistency with the least computation among sparse rules. Three aligned panels compare six inference rules on decision consistency, expensive interactions, and execution time. Gap-directed priority evaluates 16.8 edges with controller and verifier times 38.5 and 8.7 milliseconds.

pool	task $\uparrow$	calls $\downarrow$	ctrl. ms $\downarrow$	memory GB $\downarrow$
4	0.821	1.48	5.6	15.0
8	0.833	1.59	7.9	15.2
16	0.838	1.67	11.8	15.4
32	0.842	1.74	18.6	15.8
64	0.846	1.81	31.4	16.1

Appendix

Table 17. Candidate-pool scaling with sparse CEIG inference on the same test cases. Bold marks the best value per column.

Figure 6. Candidate-pool scaling of Table 17 : (a) task score, (b) accepted calls, (c) controller time, and (d) peak memory. Separate axes retain each metric’s units. Four panels plot task score, calls, controller milliseconds, and memory in gigabytes against pool sizes four through sixty-four.

pool	actions	pairs	eval. edges	fraction $\downarrow$	cert. $\uparrow$	p95 ms $\downarrow$
4	16	6	4.8	0.800	0.996	8.4
8	46	28	9.2	0.329	0.994	12.3
16	154	120	14.6	0.122	0.991	17.8
32	562	496	19.6	0.040	0.988	26.5
64	2146	2016	24.5	0.012	0.982	44.2

Appendix

Table 18. Action-space materialization under the frozen controller and unchanged bundle cap. Bold marks the smallest evaluated fraction, the highest certificate rate, and the lowest p95 controller time.

Figure 7. Action-space materialization of Table 18 : (a) declared actions, pair edges, and evaluated expensive edges per decision on a logarithmic scale and (b) certificate rate against candidate-pool size. Panel a plots three counts against pool sizes 4 to 64 on a log scale: declared actions rise from 16 to 2146 and pair edges from 6 to 2016, while evaluated edges rise from 4.8 to 24.5. Panel b shows the certificate rate decreasing from 0.996 to 0.982.

supervision	unary calls	pair calls	preview rollouts	GPU-h $\downarrow$	$N_{\rm break}$ $\downarrow$
Unary targets only	7488	0	0	3.8	580
CEIG pair targets	7488	14750	0	11.2	940
Paired preview and decision targets	7488	14750	3072	14.8	1160
Certificate-oriented full training	7488	14750	3072	16.4	1290

Appendix

Table 19. Offline supervision cost and online amortization. All rows use the same answer backbone and report the charged label-generation units. Bold marks minimum training time and break-even queries; call counts describe supervision.

preview rule	task $\uparrow$	net benefit $\uparrow$	harmful $\downarrow$	V tok. $\downarrow$	regret $\downarrow$
No preview	0.816	0.000	0.000	0.0K	0.089
Fixed schedule	0.828	0.047	0.241	118.6K	0.063
Immediate-utility gate	0.832	0.076	0.158	84.3K	0.049
Unpaired return regression	0.836	0.098	0.106	73.1K	0.038
Paired decision-value supervision	0.842	0.124	0.043	62.4K	0.021

Appendix

Table 20. Preview-value attribution under matched information, access, and supervision budgets. Harmful previews have negative paired net benefit below a predeclared tolerance. Bold marks the best and underline the second-best value among preview-using rules; no preview is the reference row.

preview rule	perm. sens. $\downarrow$	pre AUC (0.5 ideal)	post AUC $\uparrow$
Unpaired return regression	0.016	0.508	0.724
Paired decision-value supervision	0.007	0.502	0.816

Appendix

Table 21. Pre-preview leakage test for the two learned preview rules. Pre and post AUROC are measured before and after the preview is purchased; permutation sensitivity has ideal value zero and pre-preview AUROC has chance value 0.5. Bold marks the better rule per column.

target	zero-shot drop $\downarrow$	recalibr. drop $\downarrow$	ECE $\downarrow$	viol. $\downarrow$
Smaller VLM, same domain	0.009	0.000	0.044	0.019
Second VLM family	0.018	0.001	0.051	0.023
New task domain	0.027	0.002	0.059	0.028

Appendix

Table 22. Transfer of the acquisition controller. Drops are task-score decreases under zero-shot and recalibration-only transfer; lower is better in every column, and bold marks the best value per column.

decision	issued	dense agr. $\uparrow$	disagr. $\downarrow$	regret $\downarrow$
Acquisition	222	0.991	0.009	0.003
Preview	108	0.991	0.009	0.002
STOP	118	0.992	0.008	0.002
ABSTAIN	10	1.000	0.000	0.000
All decisions	458	0.992	0.008	0.003

Appendix

Table 23. Dense-replay audit by decision type. Bold marks the best typewise agreement, disagreement, and regret; the pooled row is descriptive.

stratum	$N$	task $\uparrow$	calls $\downarrow$	V tok. $\downarrow$	95% CI
Domain 1	32	0.845	1.72	283.4K	[0.824, 0.866]
Domain 2	32	0.839	1.76	289.8K	[0.818, 0.860]
Domain 3	32	0.846	1.71	281.2K	[0.825, 0.867]
Domain 4	32	0.838	1.77	291.6K	[0.817, 0.859]
Low candidate count	64	0.837	1.54	254.3K	[0.821, 0.853]
High candidate count	64	0.847	1.94	318.7K	[0.831, 0.863]

Appendix

Table 24. Stratified held-out performance of the full controller. Intervals use the same hierarchical bootstrap as the primary analysis. Bold marks the best value within each domain, candidate-count, or interaction group.

Figure 8. Stratified held-out task scores of Table 24 with hierarchical-bootstrap 95% intervals. The shaded band and vertical line mark the aggregate interval

[0.830,0.854]

and the aggregate score 0.842. A forest plot lists four domains, two candidate-count strata, and two interaction strata with point estimates between 0.831 and 0.856 and their 95 percent intervals, over a shaded aggregate band from 0.830 to 0.854.

field	configuration
Train / validation / test	384 / 128 / 128 cases
Four domain names	DocVQA (multi-page documents), ChartQA (complex charts/plots), SceneInspect (embodied scenes), WebUI (multimodal interfaces)
Twelve question-type definitions	12 types (key-value extraction, table/form cross-referencing, multi-page layout check, trend extrapolation, nested proportion comparison, multi-series cross-reading, spatial relation query, object state verification, viewpoint consistency, widget mapping, status verification, multi-panel consistency)
Candidate construction and annotation	Multi-resolution crop hierarchies; negative distractor and conflicting-evidence injection; triple-annotator consensus with frozen SHA-256 test receipt
Answer backbone	Qwen2.5-VL-7B-Instruct
Immutable model revision	commit b8c412f8a9e2d31c4e7f

Appendix

Table 25. Experimental configuration and evaluation specification. Case counts and decoding parameters apply to all held-out comparisons.

Evidence Selection

Evidence selection chooses the documents, passages, claims, or observations that best support an answer or decision.

See all 61 papers

The Scheduler Is a Hidden Retriever: How Asynchrony Shapes Evidence Acquisition in Search Agents
Oct 19, 2026Ruiyang Ren, Yuhao Wang, Yinan Zhang +1LLM Inference Scheduling Evidence Retrieval
Nanyang Technological University Singapore
Is This Evidence Decision-Critical? Learning to Verify Rule-Governed Decisions
Sep 30, 2026Haoyang Zhang, Jianpeng Zhao, Qi Hao +1LLM Decision-Making
State Key Laboratory of Internet of Things for Smart City and Institute of Smart City Technologies, University of Macau, Macau SAR, China
What Did the Agent Actually Do? Evidence-Grounded Oversight for Long-Horizon Agents
Oct 5, 2026Zhongxiang Sun, Jiahao Yan, Hongkang Zhao +5Software Engineering Agents Human-in-the-Loop Evaluation
Gaoling School of Artificial Intelligence, Renmin University of China · Kuaishou Technology

Oct 19, 2026J/K move · Enter open · S save

The Scheduler Is a Hidden Retriever: How Asynchrony Shapes Evidence Acquisition in Search Agents

Ruiyang Ren, Yuhao Wang, Yinan Zhang, Chunyan Miao

Nanyang Technological University Singapore

LLM Inference Scheduling Evidence Retrieval Inference-Time Search Behavioral Consistency of LLM Agents

Open paper AskPDF arXiv

Language-model agents increasingly solve complex tasks through parallel reasoning and asynchronous information gathering. Controlled replay across two model families and three benchmarks shows that delivery timing redirects search decisions made before all results arrive, despite identical initial evidence. Acquisition disagreement reaches up to 24 percentage points above same-order repeat variation. This dependence persists after all initial evidence arrives and through a second search, turning temporary differences in what agents have seen into lasting differences in what they acquire. An acquisition-regret bound with a matching construction shows that invariance at a fixed early arrival count can limit the use of evidence already received. We introduce Confluent Evidence Acquisition (CEA), which instead releases local reasoning when each subquestion's evidence is complete and allocates shared search independently of completion order. For a fixed plan and stable conditional response law, we prove joint invariance of decisions, acquisitions, and answers. With question-only plans across three benchmarks, CEA brings acquisition disagreement to within 1.0 percentage point of repeat variation and achieves accuracy comparable to complete-input controls, improving over early commitment by up to 12.6 percentage points. Replacing CEA's fixed allocation with first-ready allocation under contention adds 21.6 percentage points of acquisition disagreement, while local query-target disagreement remains near repeat variation. Timed comparisons show that CEA reaches decisions earlier and responds faster than the same program waiting for all initial evidence.

Abstract

Evidence Selection

Explore similar work

The Scheduler Is a Hidden Retriever: How Asynchrony Shapes Evidence Acquisition in Search Agents

Is This Evidence Decision-Critical? Learning to Verify Rule-Governed Decisions

What Did the Agent Actually Do? Evidence-Grounded Oversight for Long-Horizon Agents