Non-Markovian RL

RL: Reinforcement Learning

Follow this topic Ask about this topic

Latest papers

15 papers3 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

RM-VLM: From Natural-Language Task Descriptions to Vision-Grounded Reward Machines
Nov 5, 2026Roko Parac, Christos Ziakas, Leo Ardon +1Reward Machines Reinforcement Learning
Imperial College London, London, United Kingdom
Robust Reward Machine Learning and Execution in Noisy Environments
Oct 31, 2026Roko Parac, Lorenzo Nodari, Daniel Furelos-Blanco +3Non-Markovian RL Learning with Noisy Labels
Imperial College London, United Kingdom · University of Brescia, Italy
RECREG: Recovery-based Shielding for Regular Safety Properties in Reinforcement Learning
Oct 16, 2026Alexander W. Goodall, Francesco BelardinelliRL Shielding Safe RL
Department of Computer Science Imperial College London
Reinforcement Learning with Hawkes-Structured Networks for Market Making
Oct 12, 2026Fernando Acero, Philipp Plank, Yufei ZhangMarket Making Hawkes Processes
Department of Mathematics, Imperial College London, United Kingdom
Exact Distinguishability in Non-Markovian Decision Processes
Oct 1, 2026Kabir Murjani, Nisarg PatelNon-Markovian RL
Nirma University Ahmedabad, India · Google Mountain View, CA
Adaptive Agent Design
Sep 11, 2026Raj Kiriti Velicheti, Subhonmesh Bose, Tamer BaşarReinforcement Learning Q-Learning
Department of Electrical and Computer Engineering and the Coordinated Science Laboratory at the University of Illinois, Urbana-Champaign, Urbana, IL 61801
Temporal-Causal Inference for Reinforcement Learning via Automata Learning
Sep 7, 2026Jan Corazza, Daniil Kaminskyi, Simon Lutz +4Reinforcement Learning Automata Learning
Research Center Trustworthy Data Science and Security, TU Dortmund University, Dortmund, 44227, Germany. · School for Engineering of Matter, Transport, and Energy at Arizona State University, Tempe, Az 85281, USA.
Pareto Q-Learning with Reward Machines
Jun 17, 2026Arnaud Lequen, Clément Legrand-Lixon, Léo SaulièresMulti-Objective Reinforcement Learning Reward Machines
Linköping University, Sweden · Univ. Lille, CNRS, Centrale Lille, UMR 9189 CRIStal, F-59000 Lille, France · Univ. Toulouse, INRAE-MIAT, Toulouse, France
Commit to the Bit: Reactive Reinforcement Learning Done Right
May 27, 2026Onno Eberhard, Claire Vernade, Michael MuehlebachReinforcement Learning Partially Observable RL
Max Planck Institute for Intelligent Systems, Tübingen, Germany · University of Tübingen · University of Technology Nuremberg.
Policy Gradient Methods for Non-Markovian Reinforcement Learning
May 11, 2026Avik Kar, Siddharth Chandak, Rahul Singh +4Reinforcement Learning Policy Gradient Methods
Department of Computer Science and Automation, Indian Institute of Science, Bengaluru, India. · Department of Electrical Engineering, Stanford University, Stanford, CA, USA. · Department of Electrical and Electronics Engineering, Nanyang Technological University, Singapore. +1
Live LTL Progress Tracking: Towards Task-Based Exploration
Apr 18, 2026Noel Brindise, Cedric Langbort, Melkior OrnikReinforcement Learning Linear Temporal Logic
Coordinated Science Laboratory, University of Illinois Urbana-Champaign, Urbana, USA
Anticipatory Reinforcement Learning: From Generative Path-Laws to Distributional Value Functions
Apr 6, 2026Daniel BlochNon-Markovian RL Reinforcement Learning
UNIVERSITY OF PARIS 6 & VINUNIVERSITY · Visting Professor at the College of Engineering and Computer Science, VinUniversity, Hanoi.
Prediction-Based Markov Violation Scores for Detecting Non-Markovian Observations in Reinforcement Learning
Mar 28, 2026Naveen MysoreReinforcement Learning Non-Markovian RL
Department of Electrical and Computer Engineering, University of California, Santa Barbara, USA
Reinforcement Learning with Symbolic Reward Machines
Mar 3, 2026Thomas Krug, Daniel NeiderReward Machines Non-Markovian RL
TU Dortmund University · Research Center Trustworthy Data Science and Security
Reinforcement Learning for Long-Horizon Unordered Tasks: From Boolean to Coupled Reward Machines
Oct 31, 2025Kristina Levina, Nikolaos Pappas, Athanasios Karapantelakis +2Reward Machines Reinforcement Learning
Linköping University, Linköping, Sweden · Ericsson Research, Stockholm, Sweden

Nov 5, 2026J/K move · Enter open · S save

RM-VLM: From Natural-Language Task Descriptions to Vision-Grounded Reward Machines

Roko Parac, Christos Ziakas, Leo Ardon, Alessandra Russo

Imperial College London, London, United Kingdom

Reward Machines Reinforcement Learning Multimodal Reward Modeling Non-Markovian RL LLM-Guided RL VLMs for Robotics

Open paper AskPDF arXiv

Reward machines (RMs) have been proposed as a reward formulation in reinforcement learning (RL) to capture non-Markovian tasks where credit assignment depends on history. By representing reward functions as automata, RMs expose the task's temporal structure and allow such rewards to be handled within a Markovian framework. However, they assume a labelling function, a proposition-grounding oracle for observations, which must be manually engineered for each environment, limiting their wider applicability. In this work, we address this limitation by proposing RM-VLM, a novel approach in which a large language model (LLM) generates an RM from a natural-language task description, and a vision-language model (VLM) acts as the labelling function, grounding the RM's propositions over image-based observations during RM-guided policy learning. A VLM-based value function estimator is used to assess the agent's progress towards each RM transition and derive a dense reward signal. We evaluate RM-VLM on three domains covering non-Markovian grid-world navigation, robotic manipulation, and 3D object collection. Our experimental results show that RM-VLM consistently achieves higher task success rates than existing VLM-based reward methods across all tested domains.