A defender anticipating an attacker needs a probability distribution over the attacker's next action from information available before that action occurs. Two structures complicate this: the attacker's behavioral regime persists and can switch within an episode, and each action combines an operation with state-dependent arguments whose feasible values change over time. We capture both with a hidden Markov model whose latent state is the active regime and whose regime-conditioned action law is defined on a feasibility-masked tree of executable actions. The same likelihood forms the next-action distribution and updates the regime belief once the action is seen, and a partially observed prefix contributes its resolved local factors. Estimation separates into convex logistic fits that Baum-Welch reuses when regimes are unlabeled. We derive a reach-weighted nodewise KL expression for regime information, a sequence-wise log-loss bound for switching regime paths, and a uniform fallback guarantee. On the CAGE 2 cyber defense benchmark the filtered latent state is worth 0.8 nats per step, and after a controller switch the loss begins to fall in three steps against ten for a recurrent baseline.
Figures & tables
Figure 1: (a) Stationary test set, blue-observation context, 90% intervals; each ablation removes one element of the regime posterior and the arrow gives the change in NLL. (b) Mean NLL before a controller switch, at the switch step, and in three five-step windows after it, on a log scale.
Stationary
Switch
Predictor
NLL ↓ [CI]
Top-1 ↑
ECE ↓
NLL ↓
ECE ↓
Action bigram
1.119 [1.00, 1.22]
63.8
0.037
1.946
0.077
Masked GRU
0.872 [0.76, 0.96]
70.5
0.054
1.821
0.112
Fixed state weights
1.758 [1.66, 1.85]
44.7
0.186
2.778
0.232
Policy-class posterior
1.350 [1.26, 1.43]
56.7
0.074
2.634
0.125
Posterior mode
0.958 [0.85, 1.05]
67.4
0.052
2.142
0.107
Table 1: Stationary test set ( 25 held-out episodes per policy, active steps) and switch test set. NLL in nats per step, top- 1 in percent, 10 -bin ECE; stationary NLL carries its 90% bootstrap interval and switch intervals are about ±0.3 nats. Choice models are hierarchical unless marked flat. The second group uses the blue-observation context, the third adds the previous action, and bold rows are the proposed model.
We develop a predictive-first optimisation framework for streaming hidden Markov models. Unlike classical approaches that prioritise full posterior recovery under a fully specified generative model, we assume access to regime-specific predictive models whose parameters are learned online while maintaining a fixed transition prior over regimes. Our objective is to sequentially identify latent regimes while maintaining accurate step-ahead predictive distributions. Because the number of possible regime paths grows exponentially, exact filtering is infeasible. We therefore formulate streaming inference as a constrained projection problem in predictive-distribution space: under a fixed hypothesis budget, we approximate the full posterior predictive by the forward-KL optimal mixture supported on S paths. The solution is the renormalised top-S posterior-weighted mixture, providing a principled derivation of beam search for HMMs. The resulting algorithm is fully recursive and deterministic, performing beam-style truncation with closed-form predictive updates and requiring neither EM nor sampling. Empirical comparisons against Online EM and Sequential Monte Carlo under matched computational budgets demonstrate competitive prequential performance.
Accurate action models are critical for effective planning. Existing action-model learning methods largely assume simple action representations or become computationally intractable when learning conditional and quantified effects. We present Online Hypothesis-Driven Conditional Action Model Learning (OHCAM), an online approach for learning action models with conditional and quantified effects from limited interactions with the environment. OHCAM maintains a belief over hypothesized action models and actively selects informative actions to reduce uncertainty by maximizing disagreement among competing hypotheses, while being robust to noisy observations. To enable scalability, OHCAM begins with a small set of simple action model hypotheses and expands to more complex conditions only when the current hypotheses become inconsistent with the data. Experiments on six benchmark planning domains demonstrate that OHCAM is sample efficient in learning action models that solve substantially more tasks than baselines, even with observation noise. We validate OHCAM on two tasks using a Kinova Gen3 robot, demonstrating the real-world applicability of our approach.
Jeffrey Jewett, William Solow, Sandhya Saisubramanian
As LLM-based agents increasingly browse the web on users' behalf, a natural question arises: can websites passively identify which underlying model powers an agent? Doing so would represent a significant security risk, enabling targeted attacks tailored to known model vulnerabilities. Across 14 frontier LLMs and four web environments spanning information retrieval and shopping tasks, we show that an agent's actions and interaction timings, captured via a passive JavaScript tracker, are sufficient to identify the underlying model with up to 96% F1. We formalise this attack surface by demonstrating that classifiers trained on agent actions generalise across model sizes and families. We further show that strong classifiers can be trained from few interaction traces and that agent identity can be inferred early within an episode. Injecting randomised timing delays between actions substantially degrades classifier performance, but does not provide robust protection: a classifier retrained on delayed traces largely recovers performance. We release our harness and a labelled corpus of agent traces here.
William Lugoloobi, Samuelle Marro, Jabez Magomere +2
Oxford Internet Institute, University of Oxford · Department of Engineering Science, University of Oxford