Markov Models

Follow this topic Ask about this topic

Momentum

6 papers in the last four weeks, up 100% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 76

All topics

Learning Transition Kernels of Jump-Diffusion Processes with Conditional Diffusion Models
Oct 6, 2026Yuzhen Zhao, Yating Liu, Quentin GuibertStochastic Differential Equations Conditional Distribution Estimation
CEREMADE, CNRS, Universite Paris-Dauphine, PSL, 75016 Paris, France
Conditional Flow Matching for Transport Between Markov Processes
Oct 5, 2026Syamantak Kumar, Dheeraj Nagaraj, Saptarshi Roy +1Conditional Flow Matching Domain Adaptation
The University of Texas at Austin, Department of Computer Science · Google Deepmind · Mohamed bin Zayed University of Artificial Intelligence, CMS Division +1
Marginal Log-Likelihood Increments under Dirichlet-Smoothed Markov Estimation
Sep 22, 2026Levin David SchwabMarkov Models Process Mining
Machine learning kinetics from molecular dynamics data
Sep 15, 2026Jonathan Weare, Aaron R. DinnerMarkov Models Molecular Dynamics Simulation
Courant Institute School of Mathematics, Computing, and Data Science, New York University, New York, New York 10012, United States · Department of Chemistry and James Franck Institute, University of Chicago, Chicago, Illinois 60637, United States
High-Performance Tensor Formulation of the Viterbi Algorithm for Hidden Semi-Markov Models
Sep 15, 2026Lorenzo Piarulli, Elia Belli, Daniele De SensiMarkov Models High-Performance Computing
Department of Computer Science Sapienza University of Rome
Strong and Compact Policies for Submodular Markov Decision Processes via LP-Based Submodular Orienteering
Sep 14, 2026Lars Rohwedder, Rico ZenklusenSubmodular Optimization Markov Decision Processes
University of Southern Denmark, Odense, Denmark · ETH Zurich, Zurich, Switzerland
Supermartingale Certificates for Parametric MDPs
Sep 14, 2026Kaushik Mallik, \DHorj̣e ŽikelićRobust Markov Decision Processes Markov Decision Processes
IMDEA Software Institute, Madrid, Spain
Windowed A-K-MDP
Sep 12, 2026Xiangwen Yang, Frankie Cho, Iadine ChadesMarkov Decision Processes Markov Models
Monash University Melbourne, Australia
Artificial Rosetta Stone: Constrained Maximum A Posteriori (MAP) Reconstruction of Symbolic Raga Sequences via Order-k Markov Models
Sep 1, 2026Saanvi Raghavendran, Abhishek BhattacharjeeMarkov Models Sequence Modeling
Abstract Math Institute
When Metropolis and Hastings Meet Bradley and Terry: Exact MCMC From Preference Voting
Sep 1, 2026Ariel Smogorghevski, Nir Rosenfeld, Yaniv RomanoMarkov Chain Monte Carlo Markov Models
Department of Data and Decision Sciences, Technion – Israel Institute of Technology · Department of Computer Science, Technion – Israel Institute of Technology · Department of Electrical and Computer Engineering, Technion – Israel Institute of Technology
Exact Global MCMC with Denoising Diffusion
Aug 31, 2026Mitch HillMarkov Chain Monte Carlo Markov Models
From Optimal Actions to World Models: Identifiability of Transition Kernels in Discounted MDPs
Aug 7, 2026Neal BatraMarkov Decision Processes Markov Models
Dovetail Research · University of Warwick
Newton-Schulz Retraction-Based Inference Enables Hidden Quantum Markov Models to Outperform Classical HMMs
Aug 6, 2026Ning NingNewton-Schulz Iteration Markov Models
Department of Statistics, Texas A&M University College Station, Texas, USA
Robust Average-Reward Markov Decision Processes: Minimax-Optimal Learning via Plug-in Reductions
Aug 6, 2026Yuepeng Yang, Yuxin Chen, Yuejie ChiRobust Markov Decision Processes Average-Reward RL
*Department of Statistics and Data Science, Yale University. · †Department of Statistics and Data Science, the Wharton School, University of Pennsylvania.
MACRO: Markov Chain Routing of Transformer Layers
Aug 6, 2026Paweł Batorski, Abtin Pourhadi, Akylgali Aitaza +2Markov Models LLM Inference Acceleration
Heinrich Heine University Düsseldorf · Jagiellonian University · IDEAS Research Institute
PAC Approximation and DIRECT Optimization for Parametric Markov Models
Aug 3, 2026Zhiming Chi, Ying Liu, Andrea Turrini +2Robust Markov Decision Processes Markov Decision Processes
Key Laboratory of System Software (Chinese Academy of Sciences), Beijing, China · Institute of Software, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China +2
Tensor Probabilistic Model Checking of Finite-Horizon Markov Chains (Extended Version)
Aug 1, 2026Jianlin Li, Nick Guo, Peter Ye +1Reachability Analysis Markov Models
David R. Cheriton School of Computer Science University of Waterloo, Canada
Minimal Markovization via Stable Quotients in Holonomy-Cover Decision Processes
Jul 29, 2026Zuyuan Zhang, Yongshan Chen, Mahdi Imani +1Markov Decision Processes Partially Observable RL
Department of Electrical and Computer Engineering, The George Washington University · Department of Electrical and Computer Engineering, Northeastern University
Property-driven Causal Abstractions for Markov Decision Processes
Jul 29, 2026Jule Schmidt, Maximilian Weininger, Clemens Dubslaff +2Markov Decision Processes Markov Models
Ruhr-University Bochum, Germany · Eindhoven University of Technology, The Netherlands · Oxford University, UK +1
dtControl2+ $\varepsilon$ : Trading Optimality for Explainability in MDPs via Decision Trees
Jul 28, 2026Tereza Kinská, Jan Křetínský, Tobias Meggendorfer +2Markov Decision Processes Markov Models
Masaryk University, Brno, Czech Republic · Technical University of Munich, Munich, Germany · Lancaster University Leipzig, Leipzig, Germany +1
A Characterization of the Orthocomplement of the Tangent Space of Semiparametric Markov Models
Jul 26, 2026Trung Phung, Ilya ShpitserSemiparametric Inference Markov Models
Computer Science Department, Johns Hopkins University, Baltimore, Maryland, USA
Learning Ergodic Dynamical Systems from a Finite Trajectory
Jul 24, 2026Oleksii Kachaiev, Silvia Villa, Lorenzo RosascoKoopman Operator Learning Markov Models
MaLGa center, DIMA, Università degli Studi di Genova, Genoa, Italy · Istituto Italiano di Tecnologia, Genoa, Italy · MaLGa center, DIBRIS, Università degli Studi di Genova, Genoa, Italy
Mean-to-Score Discrete Diffusion: Posterior-Mean Denoisers for Score Entropy
Jul 23, 2026Jingyuan Li, Xiaoyi Jiang, Yixuan Jiang +4Constrained Generative Modeling Markov Models
Tsinghua University · Beijing Institute of Mathematical Sciences and Applications · Wuhan University +1
Parallel Noising in Neural Markov Logic Networks
Jul 21, 2026Peter Jung, Giuseppe Marra, Ondrej KuzelkaMarkov Chain Monte Carlo Graph Generation
Czech Technical University, Prague, Czech Republic · Department of Computer Science, KU Leuven, Leuven, Belgium
End-to-End Markov State Sequence Learning for Auditory Attention Decoding
Jul 21, 2026Yushan Yashengjiang, Jie Zhang, Miao Sun +3Markov Models Neural Decoding
NERC-SLIP, University of Science and Technology of China (USTC), Hefei 230027, China · School of Information and Communication Engineering, Guangzhou Maritime University, Guangzhou 510725, China · Artificial Intelligence Research Institute, iFLYTEK Company, Ltd., Hefei 230088, China +1
Concurrent Image Understanding and Generation: Self-Correcting Coupled Markov Jump Processes
Jul 14, 2026Minh-Quan Le, Armand Comas, Alexandros Lattas +7Masked Diffusion Models Markov Models
Google · Stony Brook University · Google DeepMind
Risk-Aware General-Utility Markov Decision Processes
Jul 10, 2026Pedro P. Santos, Fábio Vital, Alberto Sardinha +1Markov Decision Processes Markov Models
INESC-ID · Instituto Superior Técnico, University of Lisbon · Pontifical Catholic University of Rio de Janeiro
What Does a Discrete Diffusion Model Learn?
Jul 6, 2026Rodrigo Casado Noguerales, Bernhard Schölkopf, Thomas Hofmann +1Markov Models Variational Inference
1ETH Zurich · 2Max Planck Institute for Intelligent Systems, Tübingen · 3ELLIS Institute Tübingen
Non-Asymptotic Error Bounds for SMC with Biased Proposals: Application to Conditional Diffusion Sampling
Jul 6, 2026Stanislas Strasman, Gabriel Victorino Cardoso, Sylvain Le Corff +2Markov Chain Monte Carlo Markov Models
SU, LPSM · Sorbonne Université and Université Paris Cité, CNRS, LPSM, F-75005 Paris, France · LPSM +2
Sample-Efficient Learning of Probabilistic Causes for Reachability in Markov Decision Processes with Probabilistic Guarantees
Jun 29, 2026Ryohei Oura, Georgios Fainekos, Hideki Okamoto +1Reachability Analysis Markov Decision Processes
Toyota Research Institute of North America, Toyota Motor North America · Frontier Research Center, Toyota Motor Corporation
Global Explanations for Multivariate Time Series Forecasting Models via $K$ -Order Markov Approximations
Jun 25, 2026Amadeo TunyiMultivariate Time Series Forecasting Surrogate Modeling
XITASO GmbH Austraße 35, 86153 Augsburg, Germany
Learning in Markovian bandits with non-observable states and constrained decision epochs
Jun 25, 2026Thomas Hira, Victor Boone, Urtzi Ayesta +1Multi-Armed Bandits Markov Models
IRIT, Université de Toulouse, CNRS, Toulouse INP, Toulouse, France · Ikerbasque-UPV/EHU, University of the Basque Country, Bilbao, Spain
Confidence Sequences for Online Statistical Model Checking of Markov Decision Processes
Jun 24, 2026Konstantin Kueffner, Tobias Meggendorfer, Maximilian Weininger +1Markov Decision Processes Uncertainty Quantification
Institute of Science and Technology Austria, Klosterneuburg, Austria · Lancaster University Leipzig, Leipzig, Germany · Ruhr-University Bochum, Bochum, Germany +1
Low-Complexity Policy Tessellations in Structured Markov Decision Processes
Jun 24, 2026Fredy PokouMarkov Decision Processes Reinforcement Learning
CRIStAL · Inria, University of Lille, CNRS, Centrale Lille Villeneuve-d’Ascq, France
Entropy Objectives in Markov Decision Processes
Jun 19, 2026S. Akshay, Raghav Goyal, Aditya Neeraje +1Markov Decision Processes Markov Models
Indian Institute of Technology Bombay, Mumbai, India. · Tata Institute of Fundamental Research, Mumbai, India.
Attractive and Repulsive Pattern Control in Sequence Generation
Jun 19, 2026Francois PachetMarkov Models Controllable Music Generation
Maturing Markov Decision Processes: Decision Making under Increasing Information and Shrinking Action Sets
Jun 17, 2026Jiaxi Liu, Aiping Yang, Yuhang Yang +4Markov Decision Processes Markov Models
Ant International · School of Economics, Sichuan University · School of Economics, Fudan University
Learning Policy from a Single Trajectory in Average-Reward Markov Decision Process
Jun 15, 2026Jongmin Lee, Ernest K. Ryu, Vaneet AggarwalMarkov Decision Processes Reinforcement Learning
Seoul National University · UCLA · Purdue University
Lyapunov-Based Sample Complexity Analysis for Weakly-Coupled MDPs
Jun 12, 2026Tianhao Wu, Matthew Zurek, Weina Wang +1Multi-Armed Bandits Robust Markov Decision Processes
Department of Industrial and Systems Engineering, University of Wisconsin-Madison · Department of Computer Sciences, University of Wisconsin-Madison · Computer Science Department, Carnegie Mellon University
A Continuous-Time Markov Chain Framework for Insertion Language Models
Jun 8, 2026Dhruvesh Patel, Benjamin Rozonoyer, Soumitra Das +3Markov Models Language Modeling
UMass Amherst · IBM Research · University of Toronto & Vijil
Personalized and Robust Proactive Robot Assistance with Uncertainty-Guided LLM Reasoning
Jun 7, 2026Alvaro Gonzalez, M. H. Hasan Shovo, Ali AyubMarkov Models Assistive Robotics
Concordia University, Montreal, Quebec H3G 1M8, Canada
Bayesian learning for the stochastic shortest path problem
Jun 3, 2026Chon Wai Ho, Sumeetpal S. Singh, Jiaqi GuoMarkov Decision Processes Reinforcement Learning
Department of Engineering, University of Cambridge, UK · School of Mathematics and Physics, University of Wollongong, Wollongong, Australia
A Doeblin-Anchored Contrastive Chart for Learning Markov Transition Kernels
Jun 1, 2026Ao XuContrastive Learning Markov Models
School of Artificial Intelligence Jilin University No. 2699, Qianjin Street, Chaoyang District Changchun 130012, China · Zhongguancun Academy Daniufang 2nd Ring Road, Haidian District Beijing 100094, China
TERRA: Task-Embedded Reasoning and Representation Architecture for Cross-Domain Applications
Jun 1, 2026Shayan ShokriMarkov Decision Processes Markov Models
Humanpath Labs Inc.
Reinforcement Learning with Pairwise Preferences in Long-Term Decision Problems
May 29, 2026Jonathan Colaço Carr, Prakash Panangaden, Doina Precup +1Pairwise Preference Learning Markov Decision Processes
Stanford University · School of Computer Science, McGill University, Montreal, Quebec, Canada · Mila - Quebec AI Institute, Montreal, Quebec, Canada +1
Why Linear Recurrent Memory Works in Partially Observable Reinforcement Learning
May 29, 2026Yike Zhao, Onno Eberhard, Malek Khammassi +2Partially Observable RL Markov Models
EPFL, Lausanne, Switzerland · Max Planck Institute for Intelligent Systems, Tübingen, Germany
libhmm: A Modern C++20 Library for Hidden Markov Models with Correct MLE Emission M-Steps
May 28, 2026Gary WolfmanParameter Estimation Markov Models
Independent Researcher
Sampling Data with Chains of Forward-Backward Diffusion Steps
May 26, 2026Hyunmo Kang, Noam Itzhak Levi, Corinna Elena Wegner +2Dynamical Systems Markov Chain Monte Carlo
Johns Hopkins University · EPFL · EPFL & University of Göttingen +1
On the Error-Correcting Effects of Stochasticity in Discrete Diffusion
May 26, 2026William Yuan, Sungwon Jeong, Amirali AghazadehMarkov Models Diffusion Sampling
Georgia Institute of Technology
Multi-Level Contextual Token Relation Modeling for Machine-Generated Text Detection
May 15, 2026Chenwang Wu, Yiuming Cheung, Bo Han +2AI-Generated Text Detection Markov Models
Department of Computer Science, Hong Kong Baptist University, Hong Kong, China · School of Software Engineering, South China University of Technology, Guangzhou, Guangdong 510000, China · School of Computer Science and Technology, University of Science and Technology of China, Hefei, Anhui 230000, China +1
Learning POMDP World Models from Observations with Language-Model Priors
May 13, 2026Valentin Six, Frederik Panse, Mathis Fajeau +7World Model Learning Markov Models
1Max Planck Institute for Intelligent Systems · University of Tübingen · 2IRIIS +3
Discrete MeanFlow: One-Step Generation via Conditional Transition Kernels
May 12, 2026Fairoz Nower Khan, Nabuat Zaman Nahim, Md Sajid Ahmed +2Markov Models MeanFlow
Department of Computer Science, University of Kentucky Lexington, Kentucky 40506, USA
Missingness-MDPs: Bridging the Theory of Missing Data and POMDPs
May 12, 2026Joshua Wendland, Markel Zubia, Roman Andriushchenko +6Markov Models Learning with Missing Data
1Ruhr University Bochum · 2Brno University of Technology · 3Radboud University Nijmegen +2
Mixing Times of Glauber Dynamics on Masked Language Models
May 11, 2026Suvadip Sana, Sami Wolf, Neer Mehta +4Dynamical Systems Markov Chain Monte Carlo
Department of Statistics and Data Science Cornell University · Cornell University · Department Cornell of University Mathematics
Attribution-based Explanations for Markov Decision Processes
May 10, 2026Paul Kobialka, Andrea Pferscher, Francesco Leofante +3Feature Attribution Markov Decision Processes
University of Oslo, Norway · Imperial College London, United Kingdom · RWTH Aachen University, Germany
Exact Regular-Constrained Variable-Order Markov Generation via Sparse Context-State Belief Propagation
May 8, 2026François PachetConstrained Generative Modeling Constrained Decoding
LIP6, Sorbonne Université · Ynosound
Multi-Environment POMDPs with Finite-Horizon Objectives
May 8, 2026Léonard Brice, Filip Cano, Krishnendu Chatterjee +2Belief-Space Planning Markov Models
Institute of Science and Technology Austria
Integrating Causal DAGs in Deep RL: Activating Minimal Markovian States with Multi-Order Exposure
May 8, 2026Jiamin Xu, Jacqueline Maasch, Kyra GanMarkov Decision Processes Causal Representation Learning
Cornell Tech, New York, NY
On-line Learning in Tree MDPs by Treating Policies as Bandit Arms
May 6, 2026Anvay Shah, Ramsundar Anandanarayanan, Sharayu Moharir +1Markov Decision Processes Markov Models
Indian Institute of Technology Bombay · Mumbai, India
Learning Time-Inhomogeneous Markov Dynamics in Financial Time Series via Neural Parameterization
May 6, 2026Jan Rovirosa, Jesse SchmolzeMarkov Models Financial Time Series
University of Wisconsin – Madison, Madison, WI, USA

Oct 6, 2026, stat.MLJ/K move · Enter open · S save

Learning Transition Kernels of Jump-Diffusion Processes with Conditional Diffusion Models

Yuzhen Zhao, Yating Liu, Quentin Guibert

CEREMADE, CNRS, Universite Paris-Dauphine, PSL, 75016 Paris, France

Stochastic Differential Equations Conditional Distribution Estimation Conditional Diffusion Models Markov Models

Open paper AskPDF arXiv

We study the problem of learning transition kernels for time-homogeneous jump-diffusion processes using conditional diffusion models, with the goal of generating new sample paths from training data consisting of N independent trajectories observed on a high-frequency discrete time grid. On the theoretical side, we establish non-asymptotic bounds for the conditional score estimation error and for the KL divergence between the laws of the true and generated discretely observed paths. On the numerical side, we first evaluate our method on synthetic data to assess the theoretical findings and benchmark its performance against the approach of Gao et al. (2025). We then apply our method to real-world data and investigate its performance on a probabilistic forecasting task.

Momentum

Related topics

Latest papers 76