Latest papers

32 papers2 papers in the last four weeks, down 33% on the four weeks before. 0.0% of all new papers.

Feb 5, 2026

Flow Matching for Offline Reinforcement Learning with Discrete Actions

Generative policies based on diffusion models and flow matching have shown strong promise for offline reinforcement learning (RL), but their applicability remains largely confined to continuous action spaces. To address a broader range of offline RL settings, we extend flow matching to a general framework that supports discrete action spaces with multiple objectives. Specifically, we replace continuous flows with continuous-time Markov chains, trained using a Q-weighted flow matching objective. We then extend our design to multi-agent settings, mitigating the exponential growth of joint action spaces via a factorized conditional path. We theoretically show that, under idealized conditions, optimizing this objective recovers the optimal policy. Extensive experiments further demonstrate that our method performs robustly across diverse settings and benchmarks, including high-dimensional control, multi-agent games, and dynamically changing preferences over multiple objectives, while outperforming traditional offline RL methods in practical multi-modal decision-making scenarios. Our discrete framework can also be applied to continuous-control problems through action quantization, providing a flexible trade-off between representational complexity and performance.
Jan 31, 2026

Persistent-Transient Policy Evaluation for Markov Chains via Minimal Peripheral Quotients

We study fixed-policy evaluation for finite Markov chains that may be reducible and periodic. Classical evaluation methods with gain and bias decomposition are not always diagnostic: the gain records only invariant Cesàro averages, while persistent phase-dependent behavior is absorbed into the bias together with genuinely transient effects. We identify the real peripheral invariant subspace K(P)\mathcal{K}(P) of the transition matrix PP as the source of this ambiguity. Quotienting by K(P)\mathcal{K}(P) is the minimal exact quotient that removes all non-decaying modes and makes the remaining dynamics strictly stable. After choosing a gauge projection ΠΠ with kernel K(P)\mathcal{K}(P), the reward admits a unique decomposition r=gΠ⋆+(I−P)vΠ⋆r = g_Π^\star + (I-P)v_Π^\star, where gΠ⋆g_Π^\star is a persistent regime profile and vΠ⋆v_Π^\star is a gauge-fixed transient component. An exact comparison with classical normalized gain and bias shows that the new pair reallocates the same information into a persistent profile and a gauge-fixed representative of the transient quotient. This decomposition reconstructs finite-horizon returns, recovers statewise average reward, admits a transient-cost interpretation, and yields a stable estimator under a generative model.