Interpretable Early Termination of Web Navigation Agents via Closed Sequential Pattern Mining
Organizations: Department of Computer Engineering San Jose State University San Jose, CA 95192
Abstract
Web navigation agents built on open-weight language models fail often and fail late, with most of the tokens spent after the trajectory has become unrecoverable. Recent work has shown that early termination is a viable cost-reduction strategy, but existing approaches produce opaque termination decisions that do not support diagnosis or human oversight. In this work we apply closed sequential pattern mining (BIDE+) to K-step prefixes of agent execution traces to produce a small, human-reviewable failure-signature library that organizes into a four-category failure taxonomy with associated repair recommendations. We use a multi-variant evaluation protocol that explicitly removes contamination artifacts (zero-step crashes and trivially separable single-step successes) so that reported detection accuracy reflects behavioral discrimination rather than trace-length distributions. We evaluate our methodology on 1,544 MiniWoB++ traces collected from three open-weight language models, achieving the best F1 at K=3 on the Exclude-errors configuration across eight methods (0.717), recovering 88% of test-set failures within three steps, and supporting an early-termination policy that saves 26.3% of tokens at 92% precision.
Figures & tables
| Method | F1@3 | F1@5 | F1@8 | F1@10 | AUC-PR@5 |
|---|---|---|---|---|---|
| BIDE Coverage (ours) | 0.661 | 0.683 | 0.678 | 0.678 | 0.729 |
| Bi-LSTM [ 9 ] | 0.630 | 0.654 | 0.648 | 0.631 | 0.741 |
| Frequency Vector | 0.596 | 0.667 | 0.649 | 0.649 | 0.716 |
| -gram | 0.562 | 0.568 | 0.571 | 0.571 | 0.736 |
| TaSPM [ 10 ] | 0.618 | 0.618 | 0.618 | 0.618 | 0.591 |
| DeepLog [ 8 ] | 0.430 | 0.500 | 0.535 | 0.529 | 0.610 |
| Threshold | Terminated | TP | FP | Precision | Recall | Success Kill Rate | Overall Savings |
|---|---|---|---|---|---|---|---|
| 0.05 | 129 | 91 | 38 | 0.705 | 0.752 | 0.306 | 54.5% |
| 0.10 | 103 | 75 | 28 | 0.728 | 0.620 | 0.226 | 46.2% |
| 0.15 | 50 | 46 | 4 | 0.920 | 0.380 | 0.032 | 26.3% |
| 0.20 | 30 | 29 | 1 | 0.967 | 0.240 | 0.008 | 16.2% |
| 0.25 | 9 | 9 | 0 | 1.000 | 0.074 | 0.000 | 4.4% |
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
| Method | F1@3 | F1@5 | F1@8 | F1@10 | AUC-PR@10 |
|---|---|---|---|---|---|
| BIDE Coverage (ours) | 0.717 | 0.677 | 0.683 | 0.612 | 0.757 |
| Frequency Vector | 0.712 | 0.717 | 0.730 | 0.730 | 0.822 |
| -gram | 0.690 | 0.706 | 0.731 | 0.744 | 0.815 |
| Bi-LSTM [ 9 ] | 0.657 | 0.713 | 0.731 | 0.765 | 0.861 |
| TaSPM [ 10 ] | 0.676 | 0.650 | 0.656 | 0.656 | 0.622 |
| Step Count (control) | 0.667 | 0.667 | 0.667 | 0.667 | 0.500 |
Web Agents
Web agents use language and vision models to navigate websites, gather information, and complete tasks through interactive interfaces.
See all 186 papers