We present a DataBrain-specific prototype that turns natural-language analytic requests into inspectable dashboards. An installable Codex plugin and standalone Agent Runtime combine schema-constrained model decisions with typed tools, persistent state, and deterministic Hooks for approval, audit, checkpointing, recovery, and failure handling. The pipeline resolves entities, discovers metadata, enforces read-only SQL, composes dashboards, and applies static checks, dynamic preflight, and browser inspection. The model proposes actions while deterministic software controls execution and records state transitions. We evaluate the workflow on frozen real-DataBrain tasks and controlled Hook faults. Strict success was 6/8 on metadata and SQL tasks: metadata selection passed 4/4, all four SQL tasks met semantic criteria, and 2/4 met the exact output-column contract. The final release passed 4/4 single-panel dashboard tasks, one two-panel task, and one existing-dashboard refinement; a parameterised task exceeded its step limit. All ten fault scenarios met their specified outcomes without unapproved external side effects. Model inference accounted for over 97% of observed runtime in every reported group. These small, DataBrain-specific results do not establish production readiness, general text-to-SQL accuracy, or an efficiency advantage over manual dashboard construction.
Figures & tables
Area
Main focus
Requirement addressed by this work
Tool-using agents
Multi-step decisions and environmental observations
Deterministic authority over DataBrain tools, approval, and state
Text-to-SQL
Schema-grounded query generation and execution accuracy
Entity and data-domain routing under company permissions
Language-to-visualisation
Analytic specifications and chart generation
Persistent panel artifacts and verified dynamic HTML
Agent orchestration
Persistence, interrupts, and human intervention
Inspectable lifecycle policies tailored to the DataBrain workflow
TABLE I: Relationship between the closest research areas and the project gap.
Code
Requirement
FR1
Accept a natural-language goal and obtain one schema-constrained model decision at a time.
FR2
Resolve DataBrain entities, discover tables, inspect schemas, and distinguish public from business data scope.
FR3
Generate bounded read-only SQL, support dry-run, and require exact approval before real execution.
Generate dynamic HTML and keep static validation, DataBrain preflight, and credentialed preview as separate gates.
FR6
Persist Agent state, resume eligible runs, and refine an existing Dashboard without unintended replacement.
TABLE II: Evidence-backed system requirements.
Fig. 1: Evaluation evidence pipeline. Frozen criteria and append-only run records precede structured checks, rubric-based browser inspection where required, and final adjudication against a declared denominator.
Fig. 2: Dual-entry architecture with deterministic authority retained by the Runtime, Hooks, tools, and artifact validators.
Hook
Boundary
Responsibility
DashboardResumeHook
Run start
Validate and load existing Dashboard state.
ToolApprovalHook
Tool boundary
Pause before real SQL, preflight, or preview, then consume the approval.
ToolLoopGuardHook
Tool boundary
Stop an unchanged duplicate call and bound table searches.
DataBrainCircuit BreakerHook
Tool error
Convert exhausted metadata gateway retries into an external block.
CheckpointHook
Checkpoint
Atomically persist serialised AgentState .
AuditHook
All events
Append redacted line-delimited JavaScript Object Notation records without controlling execution.
TABLE III: Responsibilities of the built-in governance Hooks.
Module
Main responsibility
models.py
Serialisable decisions, calls, results, phases, trace events, and run state.
runtime.py
Bounded model, tool, and observation loop with explicit run transitions.
hooks/
Approval, circuit breaking, loop protection, audit, and checkpoint policies.
tools/
Typed adapters for DataBrain access and Dashboard artifact operations.
llm/codex_subscription.py
Account-backed model process, session reuse, and decision validation.
cli.py , evaluation.py
User interaction, recovery, experiment execution, measurement, and export.
TABLE IV: Principal standalone implementation modules.
Evidence group
Role
Completed
Strict success
Wilson 95% interval
Core metadata and SQL
Four metadata and four SQL tasks
8/8
6/8, 75.0%
40.93% to 92.85%
Final single-panel Dashboard
KPI, line, bar, and stack
4/4
4/4, 100.0%
51.01% to 100.0%
Dashboard extension
Two-panel task (D06) and joint parameterised task (D05/D07)
1/2
1/2, 50.0%
9.45% to 90.55%
Existing-Dashboard refinement
Refinement task (R03)
1/1
1/1 observed
Single observation
TABLE V: Strict task outcomes. Groups are reported separately.
Fig. 3: Strict success by declared task group and observed execution time. Bars use the denominators shown on the figure. Core time is the median over eight runs, the final single-panel group uses a group mean, and D06 and D05/D07 are single-run observations.
Group
Basis
Wall time
Model share
Model attempts
Tool calls
Core
Median or mean over 8
7.18 min median
97.95%
3.625 mean
2.625 mean
Final single-panel
Mean over 4
28.93 min
99.04%
15.0 mean
14.0 mean
D06
Single run
39.91 min
99.00%
20
19
D05/D07
Single run
61.14 min
98.45%
30
30
TABLE VI: Observed end-to-end efficiency. Aggregate and single-run rows are labelled separately.
Scenario class
Outcome
Observed result
Access and gateway handling
2/2
HTTP 403 was not retried, and exhausted HTTP 504 retries entered blocked_external .
Missing schema evidence
1/1
A completed trace without the required field was strictly rejected.
Exact approval
3/3
SQL, preflight, and preview paused, and changed arguments required new approval.
Loop control
2/2
An unchanged duplicate and a fourth table search were stopped before execution.
Persistence and audit failure
2/2
Checkpoint failure blocked the run, while fail-open audit failure preserved one benign action and recorded hook_error .
TABLE VII: Grouped outcomes from the controlled Hook fault experiment.