Bisimulation Metric

Bisimulation metrics quantify the similarity between states in dynamical systems, aiming to learn robust and efficient representations by focusing on behaviorally relevant features while discarding irrelevant details. Current research emphasizes applications in reinforcement learning, particularly for improving sample efficiency, generalization, and robustness to noise in offline and online settings, often employing techniques like kernel methods, optimal transport, and transformer architectures to compute and leverage these metrics. This work holds significant promise for advancing model-based reinforcement learning, enabling more efficient training and better generalization in complex environments, and facilitating the development of more robust and interpretable AI systems.

Papers

November 9, 2024

State Chrono Representation for Enhancing Generalization in Reinforcement Learning
Jianda Chen, Wen Zheng Terence Ng, Zichen Chen, Sinno Jialin Pan, Tianwei Zhang
Reinforcement Learning State Representation Bisimulation Metric Enhancing Generalization Structured State Evolution

October 6, 2024

Bisimulation metric for Model Predictive Control
Yutaka Shimizu, Masayoshi Tomizuka
Predictive Control Model Based Reinforcement Learning Bisimulation Metric

October 2, 2024

Stable Offline Value Function Learning with Bisimulation-based Representations
Brahma S. Pavse, Yudong Chen, Qiaomin Xie, Josiah P. Hanna
Reinforcement Learning Value Function State Action Pair Bisimulation Metric Continual Offline Reinforcement Learning

June 17, 2024

An Internal Model Principle For Robots
Vadim K. Weinstein, Tamara Alshammari, Kalle G. Timperi, Mehdi Bennis, Steven M. LaValle
Non Humanoid Robot Robot Person Robotic System Path Breaking Emergence Bisimulation Metric Surprise Minimization Internal Model

June 6, 2024

Bisimulation Metrics are Optimal Transport Distances, and Can be Computed Efficiently
Sergio Calo, Anders Jonsson, Gergely Neu, Ludovic Schwartz, Javier Segovia-Aguas
Optimal Transport Markov Chain Bisimulation Metric

May 24, 2024

Bisimulation Learning
Alessandro Abate, Mirco Giacobbe, Yannik Schnitzer
Bisimulation Metric Counterexample Guided Inductive Synthesis

March 25, 2024

Learning Action-based Representations Using Invariance
Max Rudolph, Caleb Chuck, Kevin Black, Misha Lvovsky, Scott Niekum, Amy Zhang
LeArning Abstract Meaningful Representation Alpha Invariance Controllability Condition Bisimulation Metric

January 23, 2024

Building Minimal and Reusable Causal State Abstractions for Reinforcement Learning
Zizhao Wang, Caroline Wang, Xuesu Xiao, Yuke Zhu, Peter Stone
Reinforcement Learning Causal Model State Space Implicit Model Bisimulation Metric Causal Abstraction

October 31, 2023

Offline RL with Observation Histories: Analyzing and Improving Sample Complexity
Joey Hong, Anca Dragan, Sergey Levine
Offline Reinforcement Learning Sample Complexity Optimal Trajectory Optimal Behavior Bisimulation Metric Generating Future Observation

October 27, 2023

STV+Reductions: Towards Practical Verification of Strategic Ability Using Model Reductions
Damian Kurpiewski, Witold Pazderski, Wojciech Jamroga, Yan Kim
Constructive Reduction Model Reduction Bisimulation Metric Adaptive Model Strategic Ability Validation Approach Strategy Synthesis

October 26, 2023

Understanding and Addressing the Pitfalls of Bisimulation-based Representations in Offline Reinforcement Learning
Hongyu Zang, Xin Li, Leiji Zhang, Yang Liu, Baigui Sun, Riashat Islam, Remi Tachet des Combes, Romain Laroche
Reinforcement Learning Offline Reinforcement Learning Human Understanding Common Pitfall Bisimulation Metric

October 25, 2023

Towards Control-Centric Representations in Reinforcement Learning from Images
Chen Liu, Hongyu Zang, Xin Li, Yong Heng, Yifei Wang, Zhen Fang, Yisen Wang, Mingzhong Wang
Reinforcement Learning Bisimulation Metric DeepMind Control Suite Reward Free Reinforcement Learning Image Based Reinforcement Learning Control Centric Representation

July 30, 2023

Improving Probabilistic Bisimulation for MDPs Using Machine Learning
Mohammadsadegh Mohaghegh, Khayyam Salehi
Machine Learning State Space Bisimulation Metric

June 7, 2023

Generalization Across Observation Shifts in Reinforcement Learning
Anuj Mahajan, Amy Zhang
Reinforcement Learning Reinforcement Learning Agent Bisimulation Metric Covariate Shift Generalization

March 15, 2023

The Image of the Process Interpretation of Regular Expressions is Not Closed under Bisimulation Collapse
Clemens Grabmayer
Goal Vertex Bisimulation Metric Proof System Expressivity Transfer Regular Expression Process Insight

March 13, 2023

Minimizing Fuzzy Interpretations in Fuzzy Description Logics by Using Crisp Bisimulations
Linh Anh Nguyen
Description Logic Bisimulation Metric Fuzzy Interpretation

February 12, 2023

Robust Representation Learning by Clustering with Bisimulation Metrics for Visual Reinforcement Learning with Distractions
Qiyuan Liu, Qi Zhou, Rui Yang, Jie Wang
Representation Learning Visual Reinforcement Learning Sample Efficient Reinforcement Learning Robust Representation Learning Bisimulation Metric Distraction Level Behavioral Similarity

January 9, 2023

BQ-NCO: Bisimulation Quotienting for Efficient Neural Combinatorial Optimization
Darko Drakulic, Sofia Michel, Florian Mai, Arnaud Sors, Jean-Marc Andreoli
Markov Decision Process Combinatorial Optimization Problem Salesman Problem Neural Combinatorial Optimization Bisimulation Metric Heuristic Learning

July 27, 2022

Structural Similarity for Improved Transfer in Reinforcement Learning
C. Chace Ashcraft, Benjamin Stoler, Chigozie Ewulum, Susama Agarwala
Reinforcement Learning Transfer Learning Structural Similarity Bisimulation Metric EXplicit Interest Transfer Framework

July 8, 2022

Determination of action model equivalence and simplification of action model
Jingwei Li
Text Simplification Action Model Bisimulation Metric Emotion Determination Model Equivalence

Bisimulation Metric

Papers

State Chrono Representation for Enhancing Generalization in Reinforcement Learning

Bisimulation metric for Model Predictive Control

Stable Offline Value Function Learning with Bisimulation-based Representations

An Internal Model Principle For Robots

Bisimulation Metrics are Optimal Transport Distances, and Can be Computed Efficiently

Bisimulation Learning

Learning Action-based Representations Using Invariance

Building Minimal and Reusable Causal State Abstractions for Reinforcement Learning

Offline RL with Observation Histories: Analyzing and Improving Sample Complexity

STV+Reductions: Towards Practical Verification of Strategic Ability Using Model Reductions

Understanding and Addressing the Pitfalls of Bisimulation-based Representations in Offline Reinforcement Learning

Towards Control-Centric Representations in Reinforcement Learning from Images

Improving Probabilistic Bisimulation for MDPs Using Machine Learning

Generalization Across Observation Shifts in Reinforcement Learning

The Image of the Process Interpretation of Regular Expressions is Not Closed under Bisimulation Collapse

Minimizing Fuzzy Interpretations in Fuzzy Description Logics by Using Crisp Bisimulations

Robust Representation Learning by Clustering with Bisimulation Metrics for Visual Reinforcement Learning with Distractions

BQ-NCO: Bisimulation Quotienting for Efficient Neural Combinatorial Optimization

Structural Similarity for Improved Transfer in Reinforcement Learning

Determination of action model equivalence and simplification of action model