Decision Transformer

Decision Transformers (DTs) represent a novel approach to offline reinforcement learning, framing decision-making as a sequence modeling problem using transformer architectures to predict optimal actions conditioned on past states, actions, and expected returns. Current research focuses on improving DTs' robustness to noisy data, enhancing their ability to handle long horizons and sparse rewards, and exploring their application in diverse domains, including robotics, clinical decision support, and job scheduling, often incorporating modifications like convolutional layers or recurrent networks (e.g., RWKV) to improve efficiency and performance. This approach offers a powerful alternative to traditional reinforcement learning methods, potentially leading to more sample-efficient and generalizable AI agents for various real-world applications.

Papers

February 6, 2024

January 29, 2024

Context-Former: Stitching via Latent Conditioned Sequence Modeling
Ziqi Zhang, Jingzehua Xu, Jinxin Liu, Zifeng Zhuang, Donglin Wang, Miao Liu, Shuai Zhang
Imitation Learning Offline Reinforcement Learning Decision Transformer Sequence Modeling Intermediate Latent Expert Trajectory Trajectory Refinement

January 22, 2024

P2DT: Mitigating Forgetting in task-incremental Learning with progressive prompt Decision Transformer
Zhiyuan Wang, Xiaoyang Qu, Jing Xiao, Bokui Chen, Jianzong Wang
Catastrophic Forgetting Faulty Negative Mitigation Decision Transformer Traditional Reinforcement Learning Task Specific Policy Task Incremental Learning

January 16, 2024

Solving Continual Offline Reinforcement Learning with Decision Transformer
Kaixin Huang, Li Shen, Chen Zhao, Chun Yuan, Dacheng Tao
Offline Reinforcement Learning Decision Transformer Offline RL Algorithm Offline Learning Continual Offline Reinforcement Learning

December 29, 2023

Self-supervised Pretraining for Decision Foundation Model: Formulation, Pipeline and Challenges
Xiaoqian Liu, Jianbin Jiao, Junge Zhang
Technical Challenge Self Supervised Pre Training Decision Transformer Self Supervised Pretraining Pipeline System New Formulation Large Scale Self Supervised

December 26, 2023

PDiT: Interleaving Perception and Decision-making Transformers for Deep Reinforcement Learning
Hangyu Mao, Rui Zhao, Ziyue Li, Zhiwei Xu, Hao Chen, Yiqun Chen, Bin Zhang, Zhen Xiao, Junge Zhang, Jiangjin Yin
Reinforcement Learning Neural Network Deep Reinforcement Learning Decision Transformer

December 21, 2023

Critic-Guided Decision Transformer for Offline Reinforcement Learning
Yuanfu Wang, Chao Yang, Ying Wen, Yu Liu, Yu Qiao
Offline Reinforcement Learning Value Function Decision Transformer Critic Model Return Conditioned Supervised Learning

December 12, 2023

Real-time Network Intrusion Detection via Decision Transformers
Jingdi Chen, Hanhan Zhou, Yongsheng Mei, Gina Adam, Nathaniel D. Bastian, Tian Lan
Decision Transformer Causal Transformer

November 1, 2023

Rethinking Decision Transformer via Hierarchical Reinforcement Learning
Yi Ma, Chenjun Xiao, Hebin Liang, Jianye Hao
Reinforcement Learning Transformer Architecture Hierarchical Reinforcement Learning Sequential Decision Decision Transformer

October 12, 2023

Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretraining
Licong Lin, Yu Bai, Song Mei
Reinforcement Learning Transformer Megatron Decepticons Offline Reinforcement Learning Decision Transformer Decision Maker Supervised Pre Training

October 4, 2023

Decision ConvFormer: Local Filtering in MetaFormer is Sufficient for Decision Making
Jeonghye Kim, Suyoung Lee, Woojun Kim, Youngchul Sung
Transformer Based Offline Reinforcement Learning Decision Making Decision Transformer Action Prediction Local Filter

October 1, 2023

Pre-training with Synthetic Data Helps Offline Reinforcement Learning
Zecheng Wang, Che Wang, Zixuan Dong, Keith Ross
Synthetic Data Offline Reinforcement Learning Pre Training Decision Transformer Multiple Gait Offline Deep Reinforcement Learning

September 28, 2023

Uncertainty-Aware Decision Transformer for Stochastic Driving Environments
Zenan Li, Fan Nie, Qiao Sun, Fang Da, Hang Zhao
Offline Reinforcement Learning Decision Transformer Stochastic Environment

September 12, 2023

ACT: Empowering Decision Transformer with Dynamic Programming via Advantage Conditioning
Chen-Xiao Gao, Chenyang Wu, Mingjun Cao, Rui Kong, Zongzhang Zhang, Yang Yu
Decision Transformer Dynamic Programming Offline Policy Consistent Conditioning Action Generation Streaming Transformer Conditional Transformer

August 31, 2023

Multi-Objective Decision Transformers for Offline Reinforcement Learning
Abdelghani Ghanem, Philippe Ciblat, Mounir Ghogho
Offline Reinforcement Learning Action Space Decision Transformer Trajectory Data Trajectory Representation

August 25, 2023

Integrating LLMs and Decision Transformers for Language Grounded Generative Quality-Diversity
Achkan Salehi, Stephane Doncieux
Reinforcement Learning Medical LLM Human Language Decision Transformer Generative Agent Behavior Space Behavior Descriptor Generative Diversity

August 18, 2023

Learning Computational Efficient Bots with Costly Features
Anthony Kobanda, Valliappan C. A., Joshua Romoff, Ludovic Denoyer
Deep Reinforcement Learning Decision Transformer Computational Cost Effective Feature AI Bot

July 16, 2023

S2R-ViT for Multi-Agent Cooperative Perception: Bridging the Gap from Simulation to Reality
Jinlong Li, Runsheng Xu, Xinyu Liu, Baolu Li, Qin Zou, Jiaqi Ma, Hongkai Yu
Real World Simulation Study Decision Transformer Perception Datasets Multi Agent Collaborative Perception Novel Vision Transformer

July 5, 2023

Elastic Decision Transformer
Yueh-Hua Wu, Xiaolong Wang, Masashi Hamaya
Optimal Trajectory Decision Transformer Action Inference