AllPath: Layered Verification for a Deployed, Self-Revising LLM Trading Agent under Human-Gated and Autonomous Regimes

Tian Zhang; Meng Li

cs.AIOct 19, 2026

AllPath: Layered Verification for a Deployed, Self-Revising LLM Trading Agent under Human-Gated and Autonomous Regimes

Authors: Tian Zhang, Meng Li

Organizations: Independent Researcher USA

Abstract

LLM trading agents are evaluated almost exclusively in backtests and autonomous benchmark harnesses. Deployment, where actions are irreversible, raises a different question: who verifies the agent's actions, and which failures can each verifier structurally catch? We report on AllPath, a personal LLM trading agent deployed against a live paper-brokerage account. The system watches positions through a deterministic rules engine, and each night it reviews its own day and proposes revisions to its own trading strategies. Verification is split across three layers. Code verifies what must not depend on judgment: deterministic gates enforce risk limits and block any strategy revision that fails an integrity check. The agent verifies itself within fixed bounds: its nightly review has an iteration cap and no tools that can act. A human verifies judgment, through approval screens that show the current state of each proposal. We also compared two verification policies for the agent's self-revisions on the live system: two and a half weeks with every self-revision human-gated, then five weeks with reflection revisions auto-applied behind the same code gates, while the operator's chat channel stayed human-gated throughout. Each layer caught a different kind of failure. The code gates rejected no proposal; their role was keeping revisions consistent. The human rejected two proposals that passed every code check, both over a disagreement about the market, and later removed 11 of the 19 rules the agent had added on its own, at a median lag of eight days. The most instructive production incidents, such as strategies silently left unmonitored, passed every gate; the operator found them by hand, and the fix each time was an interface change that makes the condition visible. We contribute the layered architecture, the two-regime account, and a production incident taxonomy, none observable in a backtest.

Figures & tables

Panel of Figure 1 — Figure 1. AllPath architecture. Every path from a proposal to an irreversible action crosses at least one verification layer; colors mark the verifier class (blue: deterministic code gates; orange: agent under enforced bounds; green: human judgment). Solid blue arrows are fully autonomous paths; green arrows pass through the operator. The dashed breaker demotes the autonomous revision path back to human confirmation on excessive drawdown.

Action	Proposed by	Verified by	Gate mechanics
Stock order ( confirm auth)	Sentinel rule or chat agent	Code, then human	Risk-gate caps $\to$ pending queue $\to$ human approve (web/link), single-use token
Stock order ( auto auth, hard rule)	Sentinel rule	Code only	Risk-gate caps; one-shot rule state; drawdown breaker as regime backstop
Option order (buy)	Sentinel rule ( auto+hard only, by grammar)	Code only	Premium + exposure caps; expiry safety sweep; breaker
Option close	Sentinel rule / expiry sweep	Code only	Never value-blocked (risk-reducing); daily count only
Strategy revision (human-gated regime)	Reflection or chat agent	Code, then human	Parse + id/version/freeze checks at propose; byte-exact staleness + re-checks at apply; superseded displacement of stale chat drafts; human reads regenerated diff
Strategy revision (autonomous regime)	Reflection	Code only	Same mechanical gates, auto-apply; scoped to reflection-sourced rows; breaker backstop

Table 1. Every action class in the system, decomposed by proposer, verifier, and gate mechanics.

Panel of Figure 3 — Figure 3. Every strategy revision in the data window (Aug 11 to Oct 2, both accounts), by day, source, and outcome. Regime B auto-applies reflection -sourced proposals only; the chat channel stays human-gated throughout, which gives a within-regime contrast.

When	Action	Proposer	Verifier	Outcome
Aug 10–Sep 28	12 stock orders under confirm authorization	Sentinel / chat	Code $\to$ human	11 $\checkmark$ executed or recorded via the risk gate; 1 rejected
Aug 11	NVDA revision v1 $\to$ v2: unit-inconsistency fix ( <12 vs. 0.2 ), momentum-add rule removed	Reflection	Code $\to$ human	$\checkmark$ applied; inert position-weight caps made effective
Aug 21, 27	Near-market add tranches after run-ups (TSM $440 step; NVDA gap-retest <$ 215 with cap raises)	Reflection	Code $\to$ human	Vetoed on the operator’s cyclicality prior; mechanically valid
Aug 28–Oct 2	56 strategy revisions across both accounts	Reflection	Code only (autonomous)	All auto-applied within about two minutes; 0 integrity-gate rejections
Aug 28–Oct 2	57 strategy revisions	Chat	Code $\to$ human	53 $\checkmark$ ; 1 rejected; 3 superseded
Sep 8	3 stale chat drafts displaced by newer proposals	Chat	Code (displacement)	Relabeled superseded ; approval tokens invalidated

Table 2. Representative cases from the deployment window: every consequential action, by proposer, verifier, and outcome.

\checkmark

= approved/executed.

Incident	Caught by	Design rule derived
Draft strategies silently excluded from monitoring; no surface said so	Operator, production	Any state that disables monitoring wears a persistent badge on every surface
Submission timestamp presented as fill time (17 h misread)	Operator, production	Submitted and filled are distinct events with distinct labels, everywhere
Id-less HTTP-200 broker payload journaled as “submitted”; 5 one-shot rules burned	Operator, production (autonomous window)	Validate response content, not shape, before recording success
External CLI dependency vanished; 2.5 h option-close retry storm while the stock leg had already exited	Code (contained), production	Preflight external dependencies; alert distinctly from routine order errors
Quote feed returned non-finite previous_close	Code gate, production	Strict ingestion validation; loud isolated failure beats silent NaN comparison
Unparseable sentinel analysis	Code default, production	Fail closed to human review; never default-execute on unparseable agent output

Table 3. Incident taxonomy (selection from 20 catalogued). “Adversarial review” = caught pre-merge during construction (§ 4.4 ); “production” = surfaced live during the deployment window.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Incident	Caught by	Design rule derived
Degenerate action amount crashes strategy loading	Adversarial review	Domain-typed parse boundary, never a bare stdlib exception
Condition-evaluator recursion DoS / non-finite literals	Adversarial review	Depth cap + finite-number constraint on any expression language, even agent-authored
TOCTOU double-execute race on order claim	Adversarial review	Atomic DB-level claim on every irreversible-action gate
Post-claim execution error misreported as "queued"	Adversarial review	Split try/except; every terminal state truthfully recorded
Stale-revision silently reverts an approved change	Adversarial review (reproduced pre-merge)	Byte-exact re-check of live file state at approval time, not the proposal’s snapshot
Draft strategies never monitored, no signal	Operator, production	Persistent, unmissable badge on any state that disables monitoring

Appendix

Table 4. All twenty catalogued incidents, in the order they were recorded.

Field	Rendered value
Strategy revision card
Header	revise strategy $\cdot$ NVDA #2 $\cdot$ reflection
Rationale	“NVDA has re-rated: 140 is now 23% below spot and would only fire after a crash we could not act on. Tightening the hard stop to 150 keeps the thesis intact while capping drawdown near 18%.”
Diff, current vs. proposed	version: 3 $\to$ 4 ; stop rule price < 140 $\to$ price < 150 ; unchanged lines collapsed (“… 5 unchanged lines”)
Frozen fields shown	status: active , authorization: confirm (unchanged by construction)
Controls	Approve; Reject with optional reason

Appendix

Table 5. Fields of an approval card (demo data). The diff is regenerated against the current file at render time; the order card’s live price and deviation are recomputed on every view.

LLM Agent Verification

LLM agent verification checks whether agents’ actions, evidence, policies, and completion claims satisfy explicit requirements.

See all 131 papers

VReAct: Verification as an Architectural Layer for LLM Agents: A V-Model Design, and a Pilot Study of Its Deterministic Core
Sep 25, 2026Ali Afoud, Jie JW WuLLM Agent Verification LLM Agent Reliability
Independent Researcher Canada · Michigan Technological University Houghton, USA
The LLM Proposes, the Executive Disposes: A Self-Verifying Agent Instrument that Dissociates Commitment Drift from Binding Drift in Long-Horizon Agents
Aug 4, 2026Mohsen ArjmandiAgent Failure Analysis AI Agent Reliability
Independent researcher
Operating-Layer Controls for Onchain Language-Model Agents Under Real Capital
Apr 28, 2026T. J. Barton, Chris Constantakis, Patti Hauseman +4AI Agent Reliability LLM Agent Evaluation
DX Research Group (DXRG)

Sep 25, 2026J/K move · Enter open · S save

VReAct: Verification as an Architectural Layer for LLM Agents: A V-Model Design, and a Pilot Study of Its Deterministic Core

Ali Afoud, Jie JW Wu

Independent Researcher Canada · Michigan Technological University Houghton, USA

LLM Agent Verification LLM Agent Reliability Runtime Enforcement for AI Agents

Open paper AskPDF arXiv

Large language model (LLM) agents built on the ReAct pattern concentrate four responsibilities in one model: selecting a strategy, choosing each action, formatting it, and judging whether the result is adequate. Nothing outside the generative loop can reject its output, so an agent that cannot progress does not report failure; it runs until an external budget terminates it. We propose VReAct, which treats verification as an architectural layer for ReAct agents by adapting the software-engineering V-model: specification levels descend from requirements to individual steps, each paired with a dedicated verifier; a deterministic controller enforces every verdict; and only verification outcomes write memory. A rejection thus localizes the level that introduced the fault, and an agent halts by declining rather than by exhaustion. Each verifier separates a zero-cost deterministic gate from an optional LLM judge, so the contribution and cost of each can be measured independently. A pilot study implements the acceptance- and unit-level verifier pairs, comparing five configurations that share one executor, tool set, and scorer and differ only in verification components, on the four-hop stratum of MuSiQue with an 8B-parameter backbone. Across 47 executions, the two unverified configurations answered none of ten questions, every run ending at a step cap or provider token limit; the verified configuration without a planner answered eight of ten and abstained on the rest. Deterministic gates produced eight of nine observed corrections at zero marginal cost, and planning degraded performance once verification was present. These results characterize termination behavior, not accuracy at scale; we outline a twelve-month plan to complete and evaluate the full architecture, including the integration-level pair the pilot omits.

Abstract

LLM Agent Verification

Explore similar work

VReAct: Verification as an Architectural Layer for LLM Agents: A V-Model Design, and a Pilot Study of Its Deterministic Core

The LLM Proposes, the Executive Disposes: A Self-Verifying Agent Instrument that Dissociates Commitment Drift from Binding Drift in Long-Horizon Agents

Operating-Layer Controls for Onchain Language-Model Agents Under Real Capital