Evaluating Governed LLM-Driven ML Experimentation: Lifecycle, Provenance, and Failure Metrics
Organizations: Karlsruhe Institute of Technology, Karlsruhe, Germany
Abstract
Autonomous agents that run ML experiments are usually judged by the best score they reach. That score says nothing about how it was reached: whether the agent stayed within the files it was allowed to edit, whether failed trials were kept or silently dropped, and whether each keep or discard decision can be traced to its evidence. We propose evaluating such systems on their experimental process as well as their outcome, and we build a governed harness that makes this possible. A control plane, not the LLM that proposes changes, owns each trial's lifecycle, checks its edit scope, decides keep or discard, and appends one record per trial to an append-only ledger. Governance metrics such as the invalid-action rate, the repeated-bad rate and provenance completeness are computed from that ledger. Across six ML tasks, from synthetic logistic regression to language-model training, and 1,445 trials, the same contract applied unchanged. The ledger exposed failures that a task score hides: a no-memory arm in which all 120 trials failed, a benchmark adapter whose bounded proposals ran out, and a reset failure that reproducibility hashes revealed, after which the affected campaigns were rerun. In a matched counterfactual, a loop that logs only successful trials left no record of 43 of 50 trials, while the governed ledger recorded all 50. Memory ablations show that memory's effect on repeated bad proposals depends on the task. We release the ledgers of all 1,445 trials, so that every number reported for them can be recomputed.
Figures & tables
| Desideratum | Operational meaning |
|---|---|
| D1. Bounded edit scope | The agent cannot change protected benchmark, data, or evaluation files. |
| D2. Complete attempt accounting | Every proposed attempt is recorded, including invalid proposals and crashes. |
| D3. Independent adjudication | Acceptance or rejection is decided by fixed control-plane rules, not by the agent. |
| D4. Provenance linkage | Each recorded attempt links to patch, configuration, run, metric, and decision evidence. |
| D5. Recomputability | Aggregate governance claims can be recomputed from the ledger without re-running training. |
| Evaluation concern | Harness mechanism | Reported metric |
|---|---|---|
| Repeated poor behaviour | Append-only memory + repeated-bad detector | Repeated-bad rate |
| No ground truth for intermediate choices | Decision authority separated from manager; provenance IDs per trial | Provenance completeness |
| Deployment and scope safety | NodeSpec whitelist, no-op guard, pending-trial guard | Invalid-action rate |
| Long-horizon failure accumulation | State machine; terminal kept / disc. / failed_inv. | Failure taxonomy |
| Benchmark reproducibility | NodeSpec YAML, deterministic patch bridge, seeded replicates | Artifact manifest |
| Node | Task type | Worker | Metric | Dir | Purpose in evaluation |
|---|---|---|---|---|---|
| resnet_trigger | Waveform classification | Claw | val_auc | max | Primary case study; memory ablation substrate |
| lr_synthetic | Logistic regression | Local | val_score | max | Cross-node transfer check; dependency-free |
| openml_credit_g | Tabular classification | Local | val_auc | max | Governance transfer to public tabular node |
| openml_bank_mkt | Tabular classification | Local | val_auc | max | Governance transfer with visible task improvement |
| mlagentbench_vec. | NumPy convolution | Local | speed_score | max | Benchmark compatibility probe |
| autoresearch_linux | LM pretraining | Claw | val_bpb | min | Real-task stress case; governance under high failure rate |
| From state | To state | Condition |
|---|---|---|
| pending | executing | Pending guard written; worker starts |
| executing | kept | Metric parses; beats current best; patch diff retained |
| executing | discarded | Metric parses; does not beat current best |
| executing | failed_invalid | Scope violation, runtime error, missing metric, no-op patch, or precondition failure |
| executing | failed_invalid | Mid-trial crash detected via stale pending guard at next campaign start |
| Terminal states: kept , discarded , failed_invalid . All three produce a complete ledger record. Recovery tool: scripts / recover \ _pending . py . | ||
| Metric | Formal definition | What it detects |
|---|---|---|
| Acceptance rate (AR) | Valid improving edits | |
| Invalid rate (IR) | Unevaluable trials | |
| Repeated-bad rate (RBR) | Repeated-bad proposals | Redundant failure modes |
| Provenance completeness | Trials with all five IDs | Records lacking the ID chain |
| Artifact-reference coverage | Patch/log refs and hashes for materialized runs; classified records for pre-run failures | Ledger-level evidence traceability |
| Failure taxonomy | Counts per FailureCategory | Classified, not hidden, failures |
| Field | Value |
|---|---|
| Kept / disc. / failed-invalid | 4 / 5 / 11 |
| Non-kept categories | 5 degraded_metric ; 11 proposal_precondition_failed |
| Acceptance / invalid rate | 0.20 / 0.55 |
| Provenance completeness | 1.00 (20/20) |
| Decision record completeness | 1.00 (20/20) |
| Failure evidence completeness | 1.00 (11/11 precondition failures classified) |
| Node | Trials | AR% | IR% | Lifecycle outcomes |
| resnet_trigger (case study) | 20 | 20.0 | 55.0 | kept, disc., failed_inv. |
| resnet_trigger (L40S, all arms) | 225 | 46.7 | 0.4 | kept, disc., failed_inv. (5 clean seeds verified) |
| lr_synthetic | 450 | 8.4 | 46.2 | kept, disc., failed_inv. |
| openml_credit_g | 230 | 13.5 | 26.1 | kept, disc., failed_inv. |
| openml_bank_marketing | 230 | 13.0 | 32.2 | kept, disc., failed_inv. |
| mlagentbench_vec. | 50 | 28.0 | 48.0 | kept, disc., failed_inv. |
| Scenario | Campaign | Governed outcome | Evidence |
|---|---|---|---|
| Out-of-scope edit (frozen file touched) | kdd_stress_scope | Patch rejected before training; classified invalid_edit_scope ; git state unchanged | 1/1 trials; patch ref retained |
| No-op patch (byte-identical edit) | kdd_stress_noop | Training skipped; classified no_op_patch ; empty patch hash recorded | 1/1 trials; ledger record present |
| Synthetic baseline: lr_synthetic (logistic regression, LocalWorker) | lr_synth_baseline | All three lifecycle outcomes in 5 trials; provenance complete; NodeSpec YAML portable | 2 kept, 1 discarded, 2 failed-invalid; best val_score = 0.923 |
| Synthetic LangGraph transfer | lr_synth_lg_* | Same manager/memory ablation runs on a different node and worker type; all data-generation constants remain frozen | 30/30 valid trials; RBR none/summary/rationale = 0.70/0.70/0.80 |
| Node (profile) | Governed | Ungoverned | Drops | Interpretation | |
| autoresearch_linux IR = 100% | 30 | 1.00 | 0.00 | 30 (100%) | Total failure: ungoverned ledger is empty; campaign indistinguishable from one that never ran |
| openml_bank_mkt IR = 55% | 20 | 1.00 | 0.35 | 13 (65%) | Mixed outcomes: ungoverned ledger retains only 7 valid trials; 13 classified failures leave no trace |
| Combined: 43 / 50 ungoverned trials absent, i.e. all failed trials of these two campaigns, which were chosen to span failure rates; 0 / 50 governed trials absent (100% completeness by structural invariant). | |||||
| Claim | Evidence level | Where |
|---|---|---|
| Control plane classifies kept / discarded / failed_invalid | Demonstrated | Secs 5.1–5.6 |
| Append-only ledger: 100% provenance | Demonstrated | All nodes; Sec 5.2 |
| Guard failures visible in audit record | Demonstrated | Stress, Sec 5.3 |
| Protocol transfers across node types | Demonstrated, 6 nodes | Secs 5.2–5.6 |
| Repeated-bad rate diagnoses memory sensitivity | Mixed case study | ResNet: summary lowest in 2/3; lr_synthetic fails |
| Memory ordering none summary rationale (AUC) | Observed, 5 clean seeds | Sec 5.5 ; not on lr_synthetic |
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
| Field | Meaning |
|---|---|
| trial_id | Stable identifier for one budget slot. |
| campaign_id | Links rows to the campaign ID inventory in Section B.1 . |
| budget_index | Trial index within the fixed campaign budget. |
| decision | Terminal state: kept , discarded , or failed_invalid . |
| failure_category | Control-plane label such as runtime_error , invalid_edit_scope , no_op_patch , proposal_precondition_failed , invalid_config , metric_missing , or edit_failed ; valid-but-worse trials use degraded_metric and are discarded . |
| proposal_summary , proposal_rationale | Manager proposal text recorded before worker execution. |
| Campaign ID(s) | Node | Arm / mode | Seed(s) | Budget |
|---|---|---|---|---|
| kdd_resnet_scientific_20 | resnet_trigger | case study | – | 20 |
| deepseek_resnet_none_s1 – s5 | resnet_trigger | none | s1–s5 | 15 |
| deepseek_resnet_append_only_summary_s1 – s5 | resnet_trigger | summary | s1–s5 | 15 |
| deepseek_resnet_append_only_summary_with_rationale_s1 – s5 | resnet_trigger | rationale | s1–s5 | 15 |
| deepseek_lr_none_b30_s1 – s5 | lr_synthetic | none | s1–s5 | 30 |
| deepseek_lr_append_only_summary_b30_s1 – s5 | lr_synthetic | summary | s1–s5 | 30 |
| Campaign ID(s) | Node | Arm / mode | Seed(s) | Budget |
|---|---|---|---|---|
| kdd_stress_scope | resnet_trigger | scope stress | – | 1 |
| kdd_stress_noop | resnet_trigger | no-op stress | – | 1 |
| lr_synth_baseline | lr_synthetic | baseline manager | – | 5 |
| lr_synth_lg_none | lr_synthetic | none | – | 10 |
| lr_synth_lg_summary | lr_synthetic | summary | – | 10 |
| lr_synth_lg_rationale | lr_synthetic | rationale | – | 10 |
| Arm | Memory mode | RBR | Kept | Disc. | Best metric | Observation |
| Primary ablation — rep1, budget=10 | ||||||
| A | none | 0.80 | 1 | 9 | 0.7747 | Baseline |
| B | summary | 0.50 | 4 | 6 | 0.8212 | Discovers TRAIN_FRACTION |
| D | full rationale | 0.80 | 1 | 9 | 0.7747 | Stuck on BATCH_SIZE |
| Verbosity ablation — same node/budget/model | ||||||
| C | short rationale (50 tok) | 0.80 | 1 | 9 | 0.7747 | Matches D, not B: length alone does not explain D |
| Memory mode | Seeds | Acc% | Inv% | Trial-1 AUC (s1, s2, s3, s4, s5) | Mean best AUC | 95% CI | ||
|---|---|---|---|---|---|---|---|---|
| none | 5 | 75 | 17.3 | 0.0 | 0.8878, 0.8891, 0.8890, 0.8872, 0.8873 | 0.899 | [0.892, 0.907] | 0.009 |
| summary | 5 | 75 | 64.0 | 1.3 | 0.8885, 0.8878, 0.8877, 0.8865, 0.8874 | 0.939 | [0.937, 0.940] | 0.002 |
| rationale | 5 | 75 | 58.7 | 0.0 | 0.8885, 0.8878, 0.8883, 0.8892, 0.8893 | 0.941 | [0.940, 0.943] | 0.002 |
| Arm | Seed | Kept | Disc | Fail | AR | IR | Gain |
|---|---|---|---|---|---|---|---|
| none | s1 | 0 | 0 | 30 | 0.00 | 1.00 | — |
| s2 | 0 | 0 | 30 | 0.00 | 1.00 | — | |
| s3 | 0 | 0 | 30 | 0.00 | 1.00 | — | |
| s4 | 0 | 0 | 30 | 0.00 | 1.00 | — | |
| summary | s1 | 4 | 17 | 9 | 0.13 | 0.30 | 0.0616 |
| s2 † | 0 | 0 | 30 | 0.00 | 1.00 | — |
| Node | Budget | Seeds | Trials | AR% | IR% | Mean best AUC | 95% CI |
|---|---|---|---|---|---|---|---|
| credit_g | b20 | 4 | 80 | 20.0 | 8.8 | 0.7671 | [0.7655, 0.7687] |
| b30 | 5 | 150 | 10.0 | 35.3 | 0.7679 | [0.7667, 0.7688] | |
| bank_mkt | b20 | 4 | 80 | 20.0 | 17.5 | 0.9295 | [0.9263, 0.9330] |
| b30 | 5 | 150 | 9.3 | 40.0 | 0.9341 | [0.9337, 0.9345] |
| ID | Name | Feats. | Task | Maj. cls. | |
|---|---|---|---|---|---|
| 31 | credit-g | 1,000 | 20 | Binary | 70% |
| 1461 | bank-marketing | 45,211 | 16 | Binary | 88% |
| Field | Type | Meaning |
|---|---|---|
| name | string | Unique node identifier used as campaign ledger key. |
| description | string | Human-readable node purpose. |
| editable_paths | list | Files the worker may edit; scope validator enforces this whitelist. |
| frozen_paths | list | Files and directories the worker must never touch; scope validator enforces this blocklist. |
| run_command | string | Shell command executed by the worker to train and produce output. |
| metric_name | string | Key expected in parsed metric output (e.g. val_auc ). |
| Category | Fires at | Guard | Representative ledger evidence |
|---|---|---|---|
| invalid_edit_scope | Pre-run | Scope validator | Campaign kdd_stress_scope , trial-001: proposal attempts to edit resnet_1d.py (frozen); scope guard rejects; decision is failed_invalid ; patch is never applied; node state is unchanged. |
| no_op_patch | Pre-run | No-op guard | Campaign kdd_stress_noop , trial-001: worker produces a patch that is byte-identical to the current train.py ; no-op guard rejects; decision is failed_invalid ; training is never executed. |
| proposal_precondition_failed | Pre-run | Precondition check | Campaign deepseek_autoresearch_linux_append_only_summary_s2 , trials 28–30: manager produces malformed proposals after repeated runtime_error failures; precondition guard fires before worker call. |
| runtime_error | Post-run | Metric/exit check | Campaigns deepseek_autoresearch_linux_none_s1 – s4 , all 120 trials: the worker exits non-zero (cause undetermined; see Section 5.6 ); provenance IDs complete in all 120 records. |
| invalid_config | Post-run | Node validator | OpenML nodes reject hyperparameter values outside their bounded configuration domains, e.g. excessive tree depth or estimator counts. The run emits a machine-readable invalid_config label, which the control plane records as failed_invalid . |
| metric_missing | Post-run | Metric parser | Occurs when a run completes (exit zero) but the log file contains no parseable metric line, e.g. a training script that writes to stderr only or silently skips the validation epoch. Decision is failed_invalid ; the raw log ref is retained for inspection. |