Decision Transformer

Decision Transformers (DTs) represent a novel approach to offline reinforcement learning, framing decision-making as a sequence modeling problem using transformer architectures to predict optimal actions conditioned on past states, actions, and expected returns. Current research focuses on improving DTs' robustness to noisy data, enhancing their ability to handle long horizons and sparse rewards, and exploring their application in diverse domains, including robotics, clinical decision support, and job scheduling, often incorporating modifications like convolutional layers or recurrent networks (e.g., RWKV) to improve efficiency and performance. This approach offers a powerful alternative to traditional reinforcement learning methods, potentially leading to more sample-efficient and generalizable AI agents for various real-world applications.

Papers

July 1, 2024

Multi-State-Action Tokenisation in Decision Transformers for Multi-Discrete Action Spaces
Perusha Moodley, Pramod Kaushik, Dhillu Thambi, Mark Trovinger, Praveen Paruchuri, Xia Hong, Benjamin Rosman
Decision Transformer

June 23, 2024

Accelerating Matrix Diagonalization through Decision Transformers with Epsilon-Greedy Optimization
Kshitij Bhatta, Geigh Zollicoffer, Manish Bhattarai, Phil Romero, Christian F. A. Negre, Anders M. N. Niklasson, Adetokunbo Adedoyin
Decision Transformer Epsilon Greedy Jacobian Computation Matrix Diagonalization

June 18, 2024

Sample-efficient Imitative Multi-token Decision Transformer for Generalizable Real World Driving
Hang Zhou, Dan Xu, Yiding Ji
Reinforcement Learning Decision Transformer Highway Driving Autonomous Driving Domain Multi Class Token Transformer

June 7, 2024

Pretraining Decision Transformers with Reward Prediction for In-Context Multi-task Structured Bandit Learning
Subhojyoti Mukherjee, Josiah P. Hanna, Qiaomin Xie, Robert Nowak
Multi Task Decision Transformer Near Optimal Policy Reward Prediction Contextual Bandit Setting Structured Bandit

June 4, 2024

Mamba as Decision Maker: Exploring Multi-scale Sequence Modeling in Offline Reinforcement Learning
Jiahang Cao, Qiang Zhang, Ziqing Wang, Jingkai Sun, Jiaxu Wang, Hao Cheng, Yecheng Shao, Wen Zhao, Gang Han, Yijie Guo, Renjing Xu
Reinforcement Learning Offline Reinforcement Learning Multi Scale Mamba in Mamba Decision Transformer Action Prediction Sequential Model Decision Maker

May 31, 2024

May 29, 2024

Multi-objective Cross-task Learning via Goal-conditioned GPT-based Decision Transformers for Surgical Robot Task Automation
Jiawei Fu, Yonghao Long, Kai Chen, Wang Wei, Qi Dou
Decision Transformer Surgical Robot Goal Conditioned Multi Objective Learning Surgical Automation

May 28, 2024

HarmoDT: Harmony Multi-Task Decision Transformer for Offline Reinforcement Learning
Shengchao Hu, Ziqing Fan, Li Shen, Ya Zhang, Yanfeng Wang, Dacheng Tao
Reinforcement Learning Offline Reinforcement Learning Optimal Policy Decision Transformer Subspace Learning Diverse Policy

May 22, 2024

Task-agnostic Decision Transformer for Multi-type Agent Control with Federated Split Training
Zhiyuan Wang, Bokui Chen, Xiaoyang Qu, Zhenhou Hong, Jing Xiao, Jianzong Wang
Multi Agent Split Learning Decision Transformer Decision Making Agent Federated Split Learning

May 20, 2024

Is Mamba Compatible with Trajectory Optimization in Offline Reinforcement Learning?
Yang Dai, Oubo Ma, Longfei Zhang, Xingxing Liang, Shengchao Hu, Mengzhu Wang, Shouling Ji, Jincai Huang, Li Shen
Offline Reinforcement Learning Trajectory Optimization Decision Transformer Mamba Based E RNN

May 13, 2024

Decision Mamba Architectures
André Correia, Luís A. Alexandre
Imitation Learning Decision Transformer Hierarchical Transformer Transformer Based Approach

May 10, 2024

Residual-based Attention Physics-informed Neural Networks for Spatio-Temporal Ageing Assessment of Transformers Operated in Renewable Power Plants
Ibai Ramirez, Joel Pino, David Pardo, Mikel Sanz, Luis del Rio, Alvaro Ortiz, Kateryna Morozovska, Jose I. Aizpurua
Transformer Megatron Decepticons Transformer Based Decision Transformer Life Prediction Hotspot Detection Power Transformer

May 8, 2024

Conversational Topic Recommendation in Counseling and Psychotherapy with Decision Transformer and Large Language Models
Aylin Gunal, Baihan Lin, Djallel Bouneffouf
Large Language Model Decision Transformer Conversational Recommendation Psychotherapy Model Counseling Conversation

April 3, 2024

Decision Transformer as a Foundation Model for Partially Observable Continuous Control
Xiangyuan Zhang, Weichao Mao, Haoran Qiu, Tamer Başar
Foundation Model Closed Loop Decision Transformer Continuous Control Partial Observability Controller Synthesis

March 29, 2024

Decision Mamba: Reinforcement Learning via Sequence Modeling with Selective State Spaces
Toshihiro Ota
Reinforcement Learning Mamba in Mamba Sequence of Sequence Sequential Decision Decision Transformer Selective State Space Sequential Decision Making Task

March 26, 2024

Retentive Decision Transformer with Adaptive Masking for Reinforcement Learning based Recommendation Systems
Siyu Wang, Xiaocong Chen, Lina Yao
Reinforcement Learning Recommendation System Decision Transformer Recommendation Task Inference Problem Stochastic Trajectory Trajectory Length

March 12, 2024

Reinforced Sequential Decision-Making for Sepsis Treatment: The POSNEGDM Framework with Mortality Classifier and Transformer
Dipesh Tamboli, Jiayu Chen, Kiran Pranesh Jotheeswaran, Denny Yu, Vaneet Aggarwal
Transformer Based Sequential Decision Decision Transformer Mortality Prediction Sepsis Prediction Sepsis Treatment PhysioNet Sepsis Challenge Paper

February 27, 2024

Temporal Logic Specification-Conditioned Decision Transformer for Offline Safe Reinforcement Learning
Zijian Guo, Weichao Zhou, Wenchao Li
Temporal Logic Safe Reinforcement Learning Decision Transformer Signal Temporal Logic Offline Safe Reinforcement Learning

February 20, 2024

Tiny Reinforcement Learning for Quadruped Locomotion using Decision Transformers
Orhan Eren Akgün, Néstor Cuevas, Matheus Farias, Daniel Garces
Reinforcement Learning Swarm Robotics Decision Transformer Resource Constrained Tiny Machine Learning Low Cost Quadruped Locomotion

Decision Transformer

Papers

Multi-State-Action Tokenisation in Decision Transformers for Multi-Discrete Action Spaces

Accelerating Matrix Diagonalization through Decision Transformers with Epsilon-Greedy Optimization

Sample-efficient Imitative Multi-token Decision Transformer for Generalizable Real World Driving

Pretraining Decision Transformers with Reward Prediction for In-Context Multi-task Structured Bandit Learning

Mamba as Decision Maker: Exploring Multi-scale Sequence Modeling in Offline Reinforcement Learning

Decision Mamba: Reinforcement Learning via Hybrid Selective Sequence Modeling

In-Context Decision Transformer: Reinforcement Learning via Hierarchical Chain-of-Thought

Multi-objective Cross-task Learning via Goal-conditioned GPT-based Decision Transformers for Surgical Robot Task Automation

HarmoDT: Harmony Multi-Task Decision Transformer for Offline Reinforcement Learning

Task-agnostic Decision Transformer for Multi-type Agent Control with Federated Split Training

Is Mamba Compatible with Trajectory Optimization in Offline Reinforcement Learning?

Decision Mamba Architectures

Residual-based Attention Physics-informed Neural Networks for Spatio-Temporal Ageing Assessment of Transformers Operated in Renewable Power Plants

Conversational Topic Recommendation in Counseling and Psychotherapy with Decision Transformer and Large Language Models

Decision Transformer as a Foundation Model for Partially Observable Continuous Control

Decision Mamba: Reinforcement Learning via Sequence Modeling with Selective State Spaces

Retentive Decision Transformer with Adaptive Masking for Reinforcement Learning based Recommendation Systems

Reinforced Sequential Decision-Making for Sepsis Treatment: The POSNEGDM Framework with Mortality Classifier and Transformer

Temporal Logic Specification-Conditioned Decision Transformer for Offline Safe Reinforcement Learning

Tiny Reinforcement Learning for Quadruped Locomotion using Decision Transformers