Partial Observability

Partial observability, where agents lack complete information about their environment's state, is a central challenge in many areas of artificial intelligence, particularly reinforcement learning. Current research focuses on developing methods to mitigate this limitation, employing architectures like recurrent neural networks, Kalman filters, and transformers, along with algorithms that incorporate uncertainty representation and efficient planning techniques under partial observation. These advancements are crucial for improving the performance and robustness of AI systems in complex real-world applications, such as autonomous driving, robotics, and traffic control, where complete information is rarely available. The ultimate goal is to enable agents to make optimal decisions despite incomplete knowledge of their surroundings.

Papers

October 17, 2024

On Diffusion Models for Multi-Agent Partial Observability: Shared Attractors, Error Bounds, and Composite Flow
Tonghan Wang, Heng Dong, Yanchen Jiang, David C. Parkes, Milind Tambe
Diffusion Model Partial Observability Error Bound General Flow New Attractor POMDP Policy

October 8, 2024

Network Topology Inference from Smooth Signals Under Partial Observability
Chuansen Peng, Hanning Tang, Zhiguo Wang, Xiaojing Shen
Graph Learning Partial Observability Network Structure Large Scale Network First Order Algorithm Smooth Signal Sparsity Regularization

September 25, 2024

Uncertainty Representations in State-Space Layers for Deep Reinforcement Learning under Partial Observability
Carlos E. Luis, Alessandro G. Bottero, Julia Vinogradska, Felix Berkenkamp, Jan Peters
Deep Reinforcement Learning Kalman Filter Partial Observability Uncertainty Representation Reinforcement Learning Architecture State Space Layer

September 16, 2024

Mitigating Partial Observability in Adaptive Traffic Signal Control with Transformers
Xiaoyu Wang, Ayal Taitler, Scott Sanner, Baher Abdulhai
Reinforcement Learning Transformer Megatron Decepticons Partial Observability Traffic Signal Control Adaptive Traffic Signal Control

September 2, 2024

An Examination of Offline-Trained Encoders in Vision-Based Deep Reinforcement Learning for Autonomous Driving
Shawan Mohammed, Alp Argun, Nicolas Bonnotte, Gerd Ascheid
Deep Reinforcement Learning Autonomous Driving Markov Decision Process Partial Observability Perception Module Exam Document

August 26, 2024

Equivariant Reinforcement Learning under Partial Observability
Hai Nguyen, Andrea Baisero, David Klee, Dian Wang, Robert Platt, Christopher Amato
Inductive Bias Inter Part Equivariance Partial Observability Actor Critic Reinforcement Learning Equivariant Filter

July 26, 2024

Online Planning in POMDPs with State-Requests
Raphael Avalos, Eugenio Bargiacchi, Ann Nowé, Diederik M. Roijers, Frans A. Oliehoek
Internal State Partial Observability C POMDPs Efficient Planning Spatial Planning Online Planning

July 22, 2024

On shallow planning under partial observability
Randy Lefebvre, Audrey Durand
Markov Decision Process Partial Observability Bias Variance Reinforcement Learning Framework Planning Horizon

July 10, 2024

Mitigating Partial Observability in Sequential Decision Processes via the Lambda Discrepancy
Cameron Allen, Aaron Kirtland, Ruo Yu Tao, Sam Lobel, Daniel Scott, Nicholas Petrocelli, Omer Gottesman, Ronald Parr, Michael L. Littman, George Konidaris
Sequential Decision Partial Observability Non Markovian Lambda Calculus Unobservable State

June 18, 2024

The Limits of Pure Exploration in POMDPs: When the Observation Entropy is Enough
Riccardo Zamboni, Duilio Cirino, Marcello Restelli, Mirco Mutti
Markov Decision Process Continuum Limit Partial Observability C POMDPs Shannon Entropy Pure Exploration Entanglement Entropy

May 27, 2024

Rethinking Transformers in Solving POMDPs
Chenhao Lu, Ruizhe Shi, Yuyao Liu, Kaizhe Hu, Simon S. Du, Huazhe Xu
Reinforcement Learning Markov Decision Process Inductive Bias Partial Observability

May 24, 2024

April 3, 2024

Decision Transformer as a Foundation Model for Partially Observable Continuous Control
Xiangyuan Zhang, Weichao Mao, Haoran Qiu, Tamer Başar
Foundation Model Closed Loop Decision Transformer Continuous Control Partial Observability Controller Synthesis

April 2, 2024

Multi-Agent Reinforcement Learning with Control-Theoretic Safety Guarantees for Dynamic Network Bridging
Raffaele Galliera, Konstantinos Mitsopoulos, Niranjan Suri, Raffaele Romagnoli
Multi Agent Reinforcement Learning Multi Agent System Dynamic Network Partial Observability Cooperative Task

April 1, 2024

Versatile Navigation under Partial Observability via Value-guided Diffusion Policy
Gengyu Zhang, Hao Tang, Yan Yan
Autonomous Driving Partial Observability Diffusion Policy Full Observability Visual Navigation Task Differentiable Planner

March 18, 2024

Reinforcement Learning from Delayed Observations via World Models
Armin Karamzade, Kyungmin Kim, Montek Kalsi, Roy Fox
Reinforcement Learning World Model Partial Observability C POMDPs Standard Reinforcement Learning Perception Latency

March 13, 2024

A Sparsity Principle for Partially Observable Causal Representation Learning
Danru Xu, Dingling Yao, Sébastien Lachapelle, Perouz Taslakian, Julius von Kügelgen, Francesco Locatello, Sara Magliacane
Causal Representation Learning Partial Observability Latent Causal Underlying Causal

February 28, 2024

Symmetry-aware Reinforcement Learning for Robotic Assembly under Partial Observability with a Soft Wrist
Hai Nguyen, Tadashi Kozuno, Cristian C. Beltran-Hernandez, Masashi Hamaya
Real Robot Partial Observability Robotic Assembly Free Wrist Mobility Peg in Hole Task Peg in Hole

February 27, 2024

When Your AIs Deceive You: Challenges of Partial Observability in Reinforcement Learning from Human Feedback
Leon Lang, Davis Foote, Stuart Russell, Anca Dragan, Erik Jenner, Scott Emmons
Reinforcement Learning Technical Challenge Human Feedback Partial Observability Partial Observation

Partial Observability

Papers

On Diffusion Models for Multi-Agent Partial Observability: Shared Attractors, Error Bounds, and Composite Flow

Network Topology Inference from Smooth Signals Under Partial Observability

Uncertainty Representations in State-Space Layers for Deep Reinforcement Learning under Partial Observability

Mitigating Partial Observability in Adaptive Traffic Signal Control with Transformers

An Examination of Offline-Trained Encoders in Vision-Based Deep Reinforcement Learning for Autonomous Driving

Equivariant Reinforcement Learning under Partial Observability

Online Planning in POMDPs with State-Requests

On shallow planning under partial observability

Mitigating Partial Observability in Sequential Decision Processes via the Lambda Discrepancy

The Limits of Pure Exploration in POMDPs: When the Observation Entropy is Enough

Rethinking Transformers in Solving POMDPs

Belief-State Query Policies for Planning With Preferences Under Partial Observability

Model-free reinforcement learning with noisy actions for automated experimental control in optics

Decision Transformer as a Foundation Model for Partially Observable Continuous Control

Multi-Agent Reinforcement Learning with Control-Theoretic Safety Guarantees for Dynamic Network Bridging

Versatile Navigation under Partial Observability via Value-guided Diffusion Policy

Reinforcement Learning from Delayed Observations via World Models

A Sparsity Principle for Partially Observable Causal Representation Learning

Symmetry-aware Reinforcement Learning for Robotic Assembly under Partial Observability with a Soft Wrist

When Your AIs Deceive You: Challenges of Partial Observability in Reinforcement Learning from Human Feedback