Markov Models

Follow this topic Ask about this topic

Momentum

6 papers in the last four weeks, up 100% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 76

All topics

Memory as a Markov Matrix: Sample Efficient Knowledge Expansion via Token-to-Dictionary Mapping
May 5, 2026Kaustubh Pethkar, Ziyang Xiong, Zuofeng Shang +1Continual Learning for LLMs Memorization in Language Models
New Jersey Institute of Technology · University of California, Berkeley
Taming the Curses of Multiagency in Robust Markov Games with Large State Space through Linear Function Approximation
May 4, 2026Jingchu Gai, Laixi ShiMulti-Agent System Optimization Markov Models
Machine Learning Department, Carnegie Mellon University. · Department of Electrical and Computer Engineering, Johns Hopkins University, MD 21286, USA.
Robust Parameter Learning for Uncertain MDPs
May 2, 2026Yannik Schnitzer, Alessandro Abate, David ParkerRobust Markov Decision Processes Parameter Estimation
University of Oxford, Oxford, United Kingdom
Reinforcement Learning with Markov Risk Measures and Multipattern Risk Approximation
May 1, 2026Andrzej Ruszczynski, Tiangang ZhangMarkov Decision Processes Reinforcement Learning
Department of Management Science and Information Systems, Rutgers University
Heterogeneous Variational Inference for Markov Degradation Hazard Models: Discretized Mixture with Interpretable Clusters
Apr 27, 2026Takato YasunoLatent Variable Models Markov Models
Digital Business Department, Yachiyo Engineering Co., Ltd.
Markov reads Pushkin, again: A statistical journey into the poetic world of Evgenij Onegin
Apr 22, 2026Angelo Maria SabatiniComputational Literary Studies Markov Models
The BioRobotics Institute Scuola Superiore Sant’Anna Pisa, Italy
Maximum Likelihood Reconstruction for Multi-Look Digital Holography with Markov-Modeled Speckle Correlation
Apr 22, 2026Xi Chen, Arian Maleki, Shirin JalaliMarkov Models Image Reconstruction
Department of Electrical and Computer Engineering, Rutgers University, Piscataway, NJ 08854 USA · Department of Statistics, Columbia University, New York, NY 10027 USA
Planning in entropy-regularized Markov decision processes and games
Apr 21, 2026Jean-Bastien Grill, Omar Darwiche Domingues, Pierre Ménard +2Value Function Estimation Markov Models
DeepMind Paris · SequeL team, Inria Lille
Neural Continuous-Time Markov Chain: Discrete Diffusion via Decoupled Jump Timing and Direction
Apr 17, 2026Jingyuan Li, Xiaoyi Jiang, Fukang Wen +5Markov Models Diffusion Language Models
2Beijing Institute of Mathematical Sciences and Applications · 3Wuhan University · 4MathonAI +1
Blazing the trails before beating the path: Sample-efficient Monte-Carlo planning
Apr 16, 2026Jean-Bastien Grill, Michal Valko, Rémi MunosMarkov Decision Processes Markov Models
SequeL team, INRIA Lille - Nord Europe, France · Google DeepMind, UK∗
Systematic Hazard Sampling: Minimal-Variance Inference for Discrete Diffusion and Flow Models
Jan 6, 2026Seunghwan Jang, Wonje Jeung, SooJean HanMarkov Models Diffusion Sampling
Department of EE KAIST Daejeon, South Korea
Emergence of Nonequilibrium Latent Cycles in Unsupervised Generative Modeling
Dec 12, 2025Marco Baiesi, Alberto RossoLatent Variable Models Latent Dynamics Modeling
Department of Physics and Astronomy, University of Padova, Via Marzolo 8, I-35131 Padova, Italy · INFN, Sezione di Padova, Via Marzolo 8, I-35131 Padova, Italy · LPTMS, CNRS, Universit´e Paris-Saclay, 91405, Orsay, France
Adaptive Resolving Methods for Markov Decision Processes with Function Approximations
May 17, 2025Jiashuo Jiang, Yinyu Ye, Yiming ZongMarkov Decision Processes Reinforcement Learning
Department of Industrial Engineering & Decision Analytics, The Hong Kong University of Science and Technology · Department of Management Science & Engineering, Stanford University
Reinforcement Learning in Switching Non-Stationary Markov Decision Processes: Algorithms and Convergence Analysis
Mar 24, 2025Mohsen Amiri, Sindri MagnússonNon-Stationary RL Markov Decision Processes
Department of Computer and System Science, Stockholm University, SE-164 25 Stockholm, Sweden
Exploiting Exogenous Structure for Sample-Efficient Reinforcement Learning
Sep 22, 2024Jia Wan, Sean R. Sinclair, Devavrat Shah +1Regret Minimization in RL Markov Decision Processes
Laboratory for Information and Decision Systems, Massachusetts Institute of Technology · Department of Industrial Engineering and Management Sciences, Northwestern University
A polynomial time algebraic solution to exact marginal inference in Markov Random Field models
Sep 25, 2017Ikhlef BecharEfficient Inference Markov Random Fields
Aix Marseille Univ, Universit´e de Toulon, CNRS, ENSAM, LSIS La Garde 83957, France

May 5, 2026, cs.LGJ/K move · Enter open · S save

Memory as a Markov Matrix: Sample Efficient Knowledge Expansion via Token-to-Dictionary Mapping

Kaustubh Pethkar, Ziyang Xiong, Zuofeng Shang, Yingcong Li

New Jersey Institute of Technology · University of California, Berkeley

Continual Learning for LLMs Memorization in Language Models Knowledge Augmentation for Language Models Markov Models Persistent Memory for Language Models Autoregressive Language Modeling+3

Open paper AskPDF arXiv

Continual incorporation of new knowledge is essential for the long-term evolution of large language models (LLMs). Existing approaches typically rely on parameter-update algorithms to mitigate catastrophic forgetting, yet they suffer from fundamental limitations: 1) forgetting is unavoidable as the amount of newly injected knowledge grows; and 2) model updates are often irreversible. As modern LLMs become increasingly expressive, it is natural to question whether large-scale weight updates are necessary for acquiring a small amount of new knowledge. In this work, we propose a principled framework that models autoregressive language generation as a Markov process over tokens, where model memory is represented by a Markov transition matrix. Under this formulation, incorporating new knowledge/tokens corresponds to extending the state space, and preserving existing transitions guarantees retention of previously learned knowledge. We then prove a sample complexity bound for incorporating new tokens via a token-to-dictionary mapping strategy. In particular, for learning the transition behavior of each new token, the required number of samples scales linearly with the number of existing tokens it is mapped to. To realize this mapping, we propose an embedding-tuning algorithm that requires minimal parameter updates and induces zero forgetting. Experimental results further demonstrate the effectiveness of our method and validate our theoretical findings.

Momentum

Related topics

Latest papers 76