Finite Horizon

Finite horizon problems in control and reinforcement learning focus on optimizing decisions over a fixed, limited time span, contrasting with infinite horizon approaches. Current research emphasizes efficient algorithms for solving these problems, particularly within model predictive control (MPC) and reinforcement learning frameworks, often employing techniques like policy gradients, Q-learning, and various neural network architectures to handle complex state and action spaces. This research is significant because finite horizon models are crucial for many real-world applications where time constraints are inherent, such as robotics, resource management, and financial trading, offering more realistic and practical solutions than infinite horizon counterparts.

Papers

September 8, 2022

Incremental Correction in Dynamic Systems Modelled with Neural Networks for Constraint Satisfaction
Namhoon Cho, Hyo-Sang Shin, Antonios Tsourdos, Davide Amato
Neural Network Finite Horizon Constraint Satisfaction Control Strategy Hard Inequality Constraint

September 7, 2022

On the Convergence of Monte Carlo UCB for Random-Length Episodic MDPs
Zixuan Dong, Che Wang, Keith Ross
Reinforcement Learning Early Stage Convergence Q Learning Monte Carlo Finite Horizon Q Function Episodic Markov Decision Process

August 19, 2022

A Physics-informed Deep Learning Approach for Minimum Effort Stochastic Control of Colloidal Self-Assembly
Iman Nodozi, Jared O'Leary, Ali Mesbah, Abhishek Halder
Optimal Control Finite Horizon Physic Informed Deep Learning Stochastic Control Scientific Benchmark

June 13, 2022

Markov Decision Processes under Model Uncertainty
Ariel Neufeld, Julian Sester, Mario Šikić
Markov Decision Process Model Uncertainty Finite Horizon Portfolio Optimization Optimal Portfolio

June 3, 2022

Joint Energy Dispatch and Unit Commitment in Microgrids Based on Deep Reinforcement Learning
Jiaju Qi, Lei Lei, Kan Zheng, Simon X. Yang
Deep Reinforcement Learning Finite Horizon DC Microgrid Application Unit Commitment Energy Dispatch

May 27, 2022

Solving infinite-horizon POMDPs with memoryless stochastic policies in state-action space
Johannes Müller, Guido Montúfar
Markov Decision Process Finite Horizon State Action Space Reward Maximization Maze Navigation Stochastic Policy

May 25, 2022

Near-Optimal Goal-Oriented Reinforcement Learning in Non-Stationary Environments
Liyu Chen, Haipeng Luo
Optimal Policy Finite Horizon Non Stationary Environment Goal Oriented Reinforcement Learning Dynamic Regret Minimization

May 16, 2022

Efficient Algorithms for Planning with Participation Constraints
Hanrui Zhang, Yu Cheng, Vincent Conitzer
Markov Decision Process Participation Constraint Polynomial Time Efficient Algorithm Finite Horizon

May 15, 2022

Reductive MDPs: A Perspective Beyond Temporal Horizons
Thomas Spooner, Rui Silva, Joshua Lockhart, Jason Long, Vacslav Glukhov
Markov Decision Process Optimal Policy Finite Horizon State Action Space Temporal Perspective

April 26, 2022

An Efficient Dynamic Sampling Policy For Monte Carlo Tree Search
Gongbo Zhang, Yijie Peng, Yilong Xu
Reinforcement Learning Monte Carlo Tree Search Finite Horizon Tree Search Sampling Tree

April 11, 2022

Settling the Sample Complexity of Model-Based Offline Reinforcement Learning
Gen Li, Laixi Shi, Yuxin Chen, Yuejie Chi, Yuting Wei
Offline Reinforcement Learning Sample Complexity Finite Horizon Model Based Offline Reinforcement Learning Infinite Horizon

March 22, 2022

Linear convergence of a policy gradient method for some finite horizon continuous time control problems
Christoph Reisinger, Wolfgang Stockinger, Yufei Zhang
Policy Gradient Proximal Gradient Finite Horizon Linear Convergence Control Problem

February 28, 2022

Pessimistic Q-Learning for Offline Reinforcement Learning: Towards Optimal Sample Complexity
Laixi Shi, Gen Li, Yuting Wei, Yuxin Chen, Yuejie Chi
Reinforcement Learning Offline Reinforcement Learning Sample Complexity Q Learning Finite Horizon Near Optimal Policy

February 22, 2022

Sequential Information Design: Markov Persuasion Process and Its Efficient Reinforcement Learning
Jibang Wu, Zixuan Zhang, Zhe Feng, Zhaoran Wang, Zhuoran Yang, Michael I. Jordan, Haifeng Xu
Efficient Reinforcement Learning Finite Horizon Bayesian Persuasion Sequential Information Design

February 14, 2022

Towards Deployment-Efficient Reinforcement Learning: Lower Bound and Optimality
Jiawei Huang, Jinglin Chen, Li Zhao, Tao Qin, Nan Jiang, Tie-Yan Liu
Reinforcement Learning Lower Bound Near Optimality Finite Horizon Near Optimal Policy Optimal Deployment

January 18, 2022

Differentially Private Reinforcement Learning with Linear Function Approximation
Xingyu Zhou
Positive Reinforcement Linear Function Approximation Finite Horizon Linear Mixture Markov Decision Process Private Reinforcement Learning

December 19, 2021

Exploration-exploitation trade-off for continuous-time episodic reinforcement learning with linear-convex models
Lukasz Szpruch, Tanut Treetanthiploet, Yufei Zhang
Model Based Reinforcement Learning Finite Horizon Episodic Reinforcement Learning Exploration Exploitation Trade Performance Gap Convex Model

December 1, 2021

DOPE: Doubly Optimistic and Pessimistic Exploration for Safe Reinforcement Learning
Archana Bura, Aria HasanzadeZonuzy, Dileep Kalathil, Srinivas Shakkottai, Jean-Francois Chamberland
Safe Reinforcement Learning Finite Horizon Simple Regret Optimistic Exploration Model Based Safe Reinforcement Learning Reward Aware

Finite Horizon

Papers

Incremental Correction in Dynamic Systems Modelled with Neural Networks for Constraint Satisfaction

On the Convergence of Monte Carlo UCB for Random-Length Episodic MDPs

A Physics-informed Deep Learning Approach for Minimum Effort Stochastic Control of Colloidal Self-Assembly

Markov Decision Processes under Model Uncertainty

Joint Energy Dispatch and Unit Commitment in Microgrids Based on Deep Reinforcement Learning

Solving infinite-horizon POMDPs with memoryless stochastic policies in state-action space

Near-Optimal Goal-Oriented Reinforcement Learning in Non-Stationary Environments

Efficient Algorithms for Planning with Participation Constraints

Reductive MDPs: A Perspective Beyond Temporal Horizons

An Efficient Dynamic Sampling Policy For Monte Carlo Tree Search

Settling the Sample Complexity of Model-Based Offline Reinforcement Learning

Linear convergence of a policy gradient method for some finite horizon continuous time control problems

Pessimistic Q-Learning for Offline Reinforcement Learning: Towards Optimal Sample Complexity

Sequential Information Design: Markov Persuasion Process and Its Efficient Reinforcement Learning

Towards Deployment-Efficient Reinforcement Learning: Lower Bound and Optimality

Differentially Private Reinforcement Learning with Linear Function Approximation

Exploration-exploitation trade-off for continuous-time episodic reinforcement learning with linear-convex models

DOPE: Doubly Optimistic and Pessimistic Exploration for Safe Reinforcement Learning