Paper ID: 2401.15719

Rates of Convergence in the Central Limit Theorem for Markov Chains, with an Application to TD Learning

R. Srikant

We prove a non-asymptotic central limit theorem for vector-valued martingale differences using Stein's method, and use Poisson's equation to extend the result to functions of Markov Chains. We then show that these results can be applied to establish a non-asymptotic central limit theorem for Temporal Difference (TD) learning with averaging.

Submitted: Jan 28, 2024