cs.LGAug 7, 2026

Sub-Quadratic Bisimulation Metrics via Approximate Nearest Neighbors: Coverage-Augmented Guarantees and Computable Two-Sided Certificates

Authors: Ibne Farabi ShihabJoyanta Jyoti Mondal

Organizations: Department of Computer Science, Iowa State University, USA · Department of Computer and Information Sciences, University of Delaware, USA

Abstract

Bisimulation metrics quantify behavioral similarity in Markov decision processes, but their Wasserstein fixed-point operator updates every state pair and incurs quadratic pairwise work. We give a certificate-carrying sub-quadratic method for MDPs with bounded transition support and a useful low-dimensional indexing representation: an approximate-nearest-neighbor index selects the pairs updated by the exact restricted operator, while monotone lower and upper runs enclose the exact metric at every sweep. The main analytical result is a coverage-augmented anytime bound: local index quality alone cannot control global error, because uncovered pairs retain their initialization gap. The limiting error is at most max(ρ,\eop/(1γ))\max(ρ,\eop/(1-γ)), and with exact covered backups the lower arm satisfies \dannd=ρ\|\dann-d\|_\infty=ρ. Because ρρ depends on the unknown exact metric, the algorithm returns the observable sandwich width instead; agreement of the induced lower and upper clusterings certifies exact recovery of the covered aggregation. A reward-oblivious lower bound shows sub-quadratic index-first coverage cannot remove the coverage term, while a separate adaptive lower bound requires Ω(\Scal)Ω(|\Scal|) pair evaluations. Exact-operator experiments verify the identity and enclosure in every seeded run, and timing experiments recover quadratic versus sub-quadratic scaling under both cheap and full Wasserstein backups. On the grouped \Scal=64|\Scal|=64 benchmark, exact restricted refinement reaches the exact-metric skyline once retrieval covers roughly half of all pairs, while independently trained MICo and DBC baselines stay 2222-33×33\times above that skyline at every retrieval budget. Taxi shows the certificate abstaining under an uninformative embedding, while a 25002500-state gridworld improves over a reward-only metric by 28.6%28.6\% using 12.8%12.8\% of one quadratic sweep.

Explore similar work

Sep 9, 2026cs.DS

A Sharp Barrier for Consistent Submodular Maximization: Any Improvement over 222-\sqrt{2} Entails Exponential Queries or Linear Recourse

Consistent submodular maximization studies the tradeoff between solution quality and stability when elements arrive over time. For a monotone submodular objective, which models diminishing returns, an algorithm maintains a set of at most kk available elements and changes only O(1)O(1) elements after each insertion. Dütting et al. [2025] established a tight 2/32/3 approximation with unrestricted computation and a polynomial-time 0.510.51 approximation. They left open at STOC 2025 whether efficient algorithms can match the offline 11/e1-1/e guarantee. We resolve this problem by proving that the supremum approximation achievable with polynomially many value queries and worst-case constant recourse is β=220.5858<11/e.β=2-\sqrt2\approx0.5858<1-1/e. For every ε>0\varepsilon>0, our randomized algorithm attains βεβ-\varepsilon with O(ε2)O(\varepsilon^{-2}) changes per insertion. Any fixed improvement requires exponentially many queries before one critical insertion or linear recourse of Ω(k)Ω(k) changes at that insertion, even with unlimited queries afterwards. This gap quantifies the cost of consistency: the current oracle hides which elements will be needed after an arrival. We also determine the exact curvature-dependent threshold 1(21)ϑ1-(\sqrt2-1)\vartheta, attain 11/eε1-1/e-\varepsilon for weighted coverage with O(ε1)O(\varepsilon^{-1}) recourse, and separate the existence of universal future-price certificates from their efficient computation. Our algorithm has a bounded-bit polynomial-time implementation for polynomial-bit rational oracle answers; the lower bound uses only logarithmic-bit rational answers.
Shi Fu, Qixin Zhang, Dacheng Tao
Aug 25, 2026cs.LG

From Relaxed Indexability to Exact Indexability: A tt-Step Approach for Partially Observable Restless Bandits

Whittle index policies offer a scalable method for restless multi-armed bandits, but under partial observability even determining the indifference subsidy at a single belief requires solving an infinite-horizon belief-state problem with no closed-form value function. Liu [10] addresses this difficulty by linearizing the unknown decision boundary, leading to a linear system and a closed-form approximate Whittle index. However, the resulting threshold uses only a one-step active--passive comparison and does not account for longer-horizon continuation values. We extend this framework to a \emph{tt-step lookahead threshold policy}. For each subsidy mm, the threshold is defined by the active-minus-passive advantage under tt-step finite-horizon value iteration. At t=1t=1, the threshold is mm-independent and recovers the linear threshold of Liu [10]; for t>1t>1, it becomes subsidy-dependent through the induced first-crossing structure and tracks the exact decision boundary more closely. The proposed algorithm does not require indexability as an input and includes an indexability verification. Under the original Whittle indexability, we prove that the tt-step approximate Whittle index converges geometrically to the exact Whittle index, W^t(ω)W(ω)=O(βt).|\widehat W_t(ω)-W(ω)|=O(β^t). Numerically, all 2,715 tested three-state instances are verified as indexable according to the proposed criterion. The P95 index error decreases from 2.18×1022.18\times10^{-2} at t=1t=1 to 8.93×1048.93\times10^{-4} at t=8t=8. In an exact-comparable instance with β=0.9999β=0.9999, t=2t=2 already recovers the exact Whittle-index ordering. Moderate-depth threshold policies also outperform the one-step baseline and remain close to the optimal dynamic-programming benchmark, while runtime grows mildly with tt.
Qizhen Jia, Keqin Liu
May 17, 2025cs.LG

Adaptive Resolving Methods for Markov Decision Processes with Function Approximations

Learning the optimal policy for Markov decision process problems (MDPs) from samples is a fundamental problem in online and data-driven decision-making. Function approximations are usually deployed to handle large or infinite state-action space. In our work, we consider the MDP problems with function approximation and we develop a new algorithm to solve it efficiently. Our algorithm is based on a linear programming (LP) reformulation and repeatedly resolves the identified reduced linear system as new transition samples arrive. After the optimal basis is identified, we show that, after NN resolving rounds, the expected averaged iterate achieves an instance-dependent O~(Cinst/N)\widetilde O(C_{\mathrm{inst}}/N) objective shortfall and signed constraint residual. We separately account for the historical samples used for basis identification and the d2d_2 transition queries used in each resolving round, which yields the corresponding total transition-query complexity. We further complement our result with a \textit{robust} O(1/N)O(1/\sqrt{N}) bound that is independent of ΔΔ. In comparison to the guarantees established in the previous literature, our instance dependent guarantee is tighter when the underlying instance is favorable, and the numerical experiments also reveal the wide applications and efficient empirical performances of our algorithms.
Jiashuo Jiang, Yinyu Ye, Yiming Zong