Non Ergodic

Non-ergodicity in stochastic systems, particularly within machine learning contexts like reinforcement learning, describes situations where the average behavior over many independent runs differs significantly from the long-term average behavior of a single run. Current research focuses on understanding and mitigating the challenges posed by non-ergodic processes, particularly in the convergence of optimization algorithms like Adam, by developing methods to transform non-ergodic systems into ergodic ones or by directly analyzing non-ergodic convergence. This is crucial because relying on ensemble averages in non-ergodic settings can lead to unreliable or even catastrophic outcomes in real-world applications, impacting the robustness and reliability of machine learning models.

Papers

October 17, 2023

Non-ergodicity in reinforcement learning: robustness via ergodicity transformations
Dominik Baumann, Erfaun Noorani, James Price, Ole Peters, Colm Connaughton, Thomas B. Schön
Reinforcement Learning Native Robustness Robust Policy Geometric Ergodicity Non Ergodic

July 20, 2023

Convergence of Adam for Non-convex Objectives: Relaxed Hyperparameters and Non-ergodic Case
Meixuan He, Yuqing Liang, Jinlan Liu, Dongpo Xu
Early Stage Convergence Non Convex Objective Non Convex Stochastic Optimization Non Ergodic

Non Ergodic

Papers

Non-ergodicity in reinforcement learning: robustness via ergodicity transformations

Convergence of Adam for Non-convex Objectives: Relaxed Hyperparameters and Non-ergodic Case