Overview

Markov models represent sequences and decisions as states linked by probabilistic transitions, making them useful when uncertainty and temporal dependence matter. Hidden Markov models use forward-backward inference and Baum–Welch estimation for unseen states, while Markov decision processes support planning and reinforcement learning; continuous-time chains also underpin discrete diffusion models. Researchers study how to learn transition kernels from limited or dependent data, build useful state abstractions, and scale inference and planning to large systems, with applications ranging from language generation to robotics and molecular dynamics. A central tension is that simple Markov states enable efficient analysis, but real systems may require richer memory to capture non-stationarity, partial observability, or slow mixing.

Latest papers

CardsList

95 papers3 papers in the last four weeks, down 40% on the four weeks before. 0.0% of all new papers.

  1. Finite speed Ornstein-Uhlenbeck-Kac processes and their applications in generative modeling

    Nov 13, 2026Richard Duong, Janis Auffenberg, Gabriele SteidlMarkov ModelsFlow-Based Generative Modeling

  2. Deep Bayesian Filtering for Nonlinear Structural System Identification: A Tutorial on Structural and Physical Inductive Biases

    Oct 27, 2026Wei Liu, Konstantinos Vlachas, Eleni ChatziSystem IdentificationNonlinear System Identification

  3. Learning Transition Kernels of Jump-Diffusion Processes with Conditional Diffusion Models

    Oct 6, 2026Yuzhen Zhao, Yating Liu, Quentin GuibertStochastic Differential EquationsConditional Distribution Estimation

  4. Conditional Flow Matching for Transport Between Markov Processes

    Oct 5, 2026Syamantak Kumar, Dheeraj Nagaraj, Saptarshi Roy +1Conditional Flow MatchingDomain Adaptation

  5. Convergence-Aware Pareto Selection of Covariate Scaling Transformations for Markov Deterioration Hazard Models: Evidence from Bridge Inspection Data

    Sep 24, 2026Takato Yasuno, Keita Kobayashi, Ryuta SakaguchiMarkov ModelsSurvival Analysis

  6. Machine learning kinetics from molecular dynamics data

    Sep 15, 2026Jonathan Weare, Aaron R. DinnerMarkov ModelsMolecular Dynamics Simulation

  7. High-Performance Tensor Formulation of the Viterbi Algorithm for Hidden Semi-Markov Models

    Sep 15, 2026Lorenzo Piarulli, Elia Belli, Daniele De SensiGPU AccelerationMarkov Models

  8. Strong and Compact Policies for Submodular Markov Decision Processes via LP-Based Submodular Orienteering

    Sep 14, 2026Lars Rohwedder, Rico ZenklusenSubmodular OptimizationMarkov Decision Processes

  9. Windowed A-K-MDP

    Sep 12, 2026Xiangwen Yang, Frankie Cho, Iadine ChadesMarkov Decision ProcessesMarkov Models

  10. Supermartingale Certificates for Parametric MDPs

    Sep 11, 2026Kaushik Mallik, Ðorđe ŽikelićRobust Markov Decision ProcessesMarkov Decision Processes

  11. When Metropolis and Hastings Meet Bradley and Terry: Exact MCMC From Preference Voting

    Sep 1, 2026Ariel Smogorghevski, Nir Rosenfeld, Yaniv RomanoMarkov Chain Monte CarloMarkov Models

  12. Propensity Straight-Through Gradients for Discrete Stochastic Systems

    Aug 26, 2026Jose M. G. Vilar, Leonor SaizStraight-Through EstimatorMarkov Chains

  13. From local kernels to global form: modeling the emergence of musical content

    Aug 25, 2026Francesco Vitucci, Michele Lorusso, Francesco ScagliolaMarkov ModelsMusic Information Retrieval

  14. A Data-Driven Approach to State Construction in Markov Models

    Aug 21, 2026Linde Van Gestel, Marie-Anne Guerry, Evy RombautMarkov ModelsState Abstraction

  15. Spectral partitioning for kk-block averaging kernels of finite Markov chains

    Aug 21, 2026Michael C. H. Choi, Youjia WangMarkov ModelsSpectral Methods

  16. On the Pseudo-Mixing of Kac's Walk

    Aug 18, 2026Natesh S. Pillai, Aaron Smith, Vinod VaikuntanathanMarkov Models

  17. Spectral Gaps of Hit-and-Run and Coordinate Hit-and-Run

    Aug 17, 2026Yunbum Kook, Santosh S. VempalaMarkov Chain Monte CarloMarkov Models

  18. Newton-Schulz Retraction-Based Inference Enables Hidden Quantum Markov Models to Outperform Classical HMMs

    Aug 6, 2026Ning NingNewton-Schulz IterationMarkov Models

  19. Robust Average-Reward Markov Decision Processes: Minimax-Optimal Learning via Plug-in Reductions

    Aug 6, 2026Yuepeng Yang, Yuxin Chen, Yuejie ChiRobust Markov Decision ProcessesAverage-Reward RL

  20. MACRO: Markov Chain Routing of Transformer Layers

    Aug 6, 2026Paweł Batorski, Abtin Pourhadi, Akylgali Aitaza +2Markov ModelsLLM Inference Acceleration

  21. PAC Approximation and DIRECT Optimization for Parametric Markov Models

    Aug 3, 2026Zhiming Chi, Ying Liu, Andrea Turrini +2Robust Markov Decision ProcessesMarkov Decision Processes

  22. Tensor Probabilistic Model Checking of Finite-Horizon Markov Chains (Extended Version)

    Aug 1, 2026Jianlin Li, Nick Guo, Peter Ye +1Reachability AnalysisMarkov Models

  23. Minimal Markovization via Stable Quotients in Holonomy-Cover Decision Processes

    Jul 29, 2026Zuyuan Zhang, Yongshan Chen, Mahdi Imani +1Markov Decision ProcessesPartially Observable RL

  24. Property-driven Causal Abstractions for Markov Decision Processes

    Jul 29, 2026Jule Schmidt, Maximilian Weininger, Clemens Dubslaff +2Markov Decision ProcessesMarkov Models