Awesome Decision Transformer

This is a collection of research papers for Decision Transformer (DT). And the repository will be continuously updated to track the frontier of DT.

Welcome to follow and star!

Table of Contents

• A Taxonomy of DT Algorithms

• Surveys

• Papers

  • Arxiv
  • ICML 2026 (New!!!)
  • ICLR 2026 (New!!!)
  • NeurIPS 2025
  • ICML 2025
  • ICLR 2025
  • NeurIPS 2024
  • IROS 2024
  • ICML 2024
  • ICLR 2024
  • NeurIPS 2023
  • CoRL 2023
  • IROS 2023
  • ICML 2023
  • ICRA 2023
  • ICLR 2023
  • NeurIPS 2022
  • CoRL 2022
  • ICML 2022
  • AAAI 2022
  • ICLR 2022
  • NeurIPS 2021
  • ICML 2021

• Contributing

Overview of Transformer

The Decision Transformer was proposed by “Decision Transformer: Reinforcement Learning via Sequence Modeling” by Chen L. et al. It casts (offline) Reinforcement Learning as a conditional-sequence modeling problem.

Specifically, DT model is a causal transformer model conditioned on the desired return, (past) states, and actions to generate future actions in an autoregressive manner.

Advantage

1. Bypass the need for bootstrapping for long term credit assignment
2. Avoid undesirable short-sighted behaviors due to the discounting future rewards.
3. Enjoy the transformer models widely used in language and vision, which are easy to scale and adapt to multi-modal data.

Surveys

• On Transforming Reinforcement Learning With Transformers: The Development Trajectory

  • Shengchao Hu, Li Shen, Ya Zhang, Yixin Chen, Dacheng Tao
  • Publisher: IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)

• Large sequence models for sequential decision-making: a survey

  • Muning Wen, Runji Lin, Hanjing Wang, Yaodong Yang, Ying Wen, Luo Mai, Jun Wang, Haifeng Zhang, Weinan Zhang
  • Publisher: Frontiers of Computer Science

• A Survey on Transformers in Reinforcement Learning

  • Wenzhe Li, Hao Luo, Zichuan Lin, Chongjie Zhang, Zongqing Lu, Deheng Ye
  • Publisher: Transactions on Machine Learning Research (TMLR)

• Transformers in Reinforcement Learning: A Survey

  • Pranav Agarwal, Aamer Abdul Rahman, Pierre-Luc St-Charles, Simon J.D. Prince, Samira Ebrahimi Kahou

Papers

format:
- [title](paper link) [links]
  - author1, author2, and author3...
  - publisher
  - key 
  - code 
  - experiment environment

Arxiv

• Decision MetaMamba: Enhancing Selective SSM in Offline RL with Heterogeneous Sequence Mixing

  • Wall Kim, Chaeyoung Song, Hanul Kim
  • Key: DT, SSM, Mamba
  • ExpEnv: D4RL

• Context-Former: Stitching via Latent Conditioned Sequence Modeling

  • Ziqi Zhang, Jingzehua Xu, Zifeng Zhuang, Jinxin Liu, Donglin wang
  • Key: DT, Latent Conditioned Sequence Modeling
  • ExpEnv: D4RL

• Real-time Network Intrusion Detection via Decision Transformers

  • Jingdi Chen, Hanhan Zhou, Yongsheng Mei, Gina Adam, Nathaniel D. Bastian, Tian Lan
  • Key: DT, Network Intrusion Detection
  • ExpEnv: UNSW-NB15

• Prompt-Tuning Decision Transformer with Preference Ranking

  • Shengchao Hu, Li Shen, Ya Zhang, Dacheng Tao
  • Key: Prompt-Tuning
  • ExpEnv: D4RL

• Graph Decision Transformer

  • Shengchao Hu, Li Shen, Ya Zhang, Dacheng Tao
  • Key: graph transformer
  • ExpEnv: Atari

• Can Offline Reinforcement Learning Help Natural Language Understanding?

  • Ziqi Zhang, Yile Wang, Yue Zhang, Donglin Wang
  • Key: Language model
  • ExpEnv: MuJoco, Maze 2D

• SaFormer: A Conditional Sequence Modeling Approach to Offline Safe Reinforcement Learning

  • Qin Zhang, Linrui Zhang, Haoran Xu, Li Shen, Bowen Wang, Yongzhe Chang, Xueqian Wang, Bo Yuan, Dacheng Tao
  • Key: Offline Safe RL, DT
  • ExpEnv: D4RL

• Offline Pre-trained Multi-Agent Decision Transformer: One Big Sequence Model Tackles All SMAC Tasks

  • Linghui Meng, Muning Wen, Yaodong Yang, Chenyang Le, Xiyun Li, Weinan Zhang, Ying Wen, Haifeng Zhang, Jun Wang, Bo Xu
  • Key: Multi-Agent RL
  • Code: official
  • ExpEnv: SMAC

• Transfer learning with causal counterfactual reasoning in Decision Transformers

  • Ayman Boustati, Hana Chockler, Daniel C. McNamee
  • Key: Causal reasoning, Transfer Learning
  • ExpEnv: MINIGRID

• Pretraining for Language Conditioned Imitation with Transformers

  • Aaron L Putterman, Kevin Lu, Igor Mordatch, Pieter Abbeel
  • Key: Text-Conditioned Decision
  • ExpEnv: Text-Conditioned Frostbite (MultiModal Benchmark)

• An Offline Deep Reinforcement Learning for Maintenance Decision-Making

  • Hamed Khorasgani, Haiyan Wang, Chetan Gupta, Ahmed Farahat
  • Publisher: Annual Conference of the PHM Society 2021
  • Key: Offline Supervised RL, Remaining Useful Life Estimation
  • ExpEnv: NASA C-MAPSS

• A Sequence Modelling Approach to Question Answering in Text-Based Games

  • Gregory Furman, Edan Toledo, Jonathan Shock, Jan Buys
  • Publisher: Proceedings of the 3rd Wordplay: When Language Meets Games Workshop (Wordplay 2022)
  • Key: VQA
  • ExpEnv: QAIT

• Can Wikipedia Help Offline Reinforcement Learning?

  • Machel Reid, Yutaro Yamada, Shixiang Shane Gu
  • Key: VLN, Transfer Learning
  • Code: official
  • ExpEnv: MuJoco, D4RL, Atari

• Switch Trajectory Transformer with Distributional Value Approximation for Multi-Task Reinforcement Learning

  • Qinjie Lin, Han Liu, Biswa Sengupta
  • Key: Multi-Task RL, Sparse Reward
  • ExpEnv: MINIGRID

• Deep Transformer Q-Networks for Partially Observable Reinforcement Learning

  • Kevin Esslinger, Robert Platt, Christopher Amato
  • Key: POMDP, Transformer Q-Learning
  • ExpEnv: GV, Car Flag

• SimStu-Transformer: A Transformer-Based Approach to Simulating Student Behaviour

  • Zhaoxing Li, Lei Shi, Alexandra Cristea, Yunzhan Zhou, Chenghao Xiao, Ziqi Pan
  • Key: Intelligent Tutoring System

• Attention-Based Learning for Combinatorial Optimization

  • Carson Smith
  • Key: Combinatorial Optimization

ICML 2026

• Decision Transformers As Zero-Shot Learners via Text-Behavior Alignment

  • Xin Zhang, Jonathan Martinez, Yanhua Li, Yingxue Zhang
  • Key: Decision Transformer, offline meta-RL, text conditioning, zero-shot
  • ExpEnv: MetaWorld, MuJoCo

• Unsupervised Partner Design Enables Robust Ad-hoc Teamwork

  • Constantin Ruhdorfer, Matteo Bortoletto, Victor Oei, Anna Penzkofer, Andreas Bulling
  • Key: Ad-hoc teamwork, In-context coordination, Multi-agent RL
  • ExpEnv: LBF, Overcooked, OvercookedAI

• Behavior-Invariant Task Representation Learning with Transformer-based World Models for Offline Meta-Reinforcement Learning

  • Fuyuan Qian, Menglong Zhang, Song Wang, Quanying Liu
  • Key: Transformer World Model, Behavior-Invariant, Offline Meta-RL, Task Representation
  • ExpEnv: offline meta-RL benchmarks

• Benchmarking the Limits of In-Context Reinforcement Learning for Ad-Hoc Teamwork

  • Yuheng Jing, Kai Li, Jiajun Zhang, Zeyao Ma, Jiaxi Yang, Lei Zhang, Zhe Wu, Jinmin He, Junliang Xing, Jian Cheng
  • Key: In-Context RL, Ad-Hoc Teamwork, Benchmark, Multi-Agent
  • ExpEnv: Overcooked-V2

• Improving Zero-Shot Offline RL via Behavioral Task Sampling

  • Nazim Bendib, Nicolas Perrin-Gilbert, Olivier Sigaud
  • Key: Zero-Shot Offline RL, Behavioral Task Sampling, Meta-RL
  • ExpEnv: offline meta-RL benchmarks

• QHyer: Q-conditioned Hybrid Attention-Mamba Transformer for Offline Goal-Conditioned RL

  • Xing Lei, Jincheng Wang, Xuetao Zhang, Donglin Wang
  • Key: Decision Transformer, Q-Conditioned, Mamba, Offline Goal-Conditioned RL
  • ExpEnv: offline goal-conditioned RL benchmarks

• Return-to-Go Is More Than a Number: Q-Guided Alignment for Return-Conditioned Supervised Learning

  • Yuxiao Yang, Weitong Zhang
  • Key: Return-to-Go, Decision Transformer, Q-Guided Alignment, Offline RL
  • ExpEnv: D4RL

• Robust In-Context Reinforcement Learning Under Reward Poisoning Attacks

  • Paulius Sasnauskas, Yiğit Yalın, Goran Radanovic
  • Key: In-Context RL, Robustness, Reward Poisoning, Transformer
  • ExpEnv: in-context RL benchmarks

ICLR 2026

• Vintix II: Decision Pre-Trained Transformer is a Scalable In-Context Reinforcement Learner

  • Andrei Polubarov, Nikita Lyubaykin, Alexander Derevyagin, Artyom Grishin, Igor Saprygin, Aleksandr Serkov, Mark Averchenko, Daniil Tikhonov, Maksim Zhdanov, Alexander Nikulin, Ilya Zisman, Albina Klepach, Alexey Zemtsov, Vladislav Kurenkov
  • Key: In-Context RL, Decision-Pretrained Transformer, Algorithm Distillation, Flow Matching, Bayesian Posterior Sampling
  • ExpEnv: multi-domain ICRL benchmarks

• Reward Is Enough: LLMs Are In-Context Reinforcement Learners

  • Kefan Song, Amir Moeini, Peng Wang, Lei Gong, Rohan Chandra, Shangtong Zhang, Yanjun Qi
  • Key: In-Context RL, LLM, Test-Time Scaling, Scalar Feedback
  • ExpEnv: LLM reasoning, creative, scientific tasks

• Scalable In-Context Q-Learning

  • Jinmei Liu, Fuhong Liu, Zhenhong Sun, Jianye Hao, Huaxiong Li, Bo Wang, Daoyi Dong, Chunlin Chen, Zhi Wang
  • Key: In-Context RL, Q-Learning, Advantage-Weighted Regression, World Model, Transformer
  • ExpEnv: ICRL benchmarks

• In-Context Learning for Pure Exploration

  • Alessio Russo, Ryan Welch, Aldo Pacchiano
  • Key: Pure Exploration, Active Sequential Hypothesis Testing, In-Context Learning, Best-Arm Identification
  • ExpEnv: best-arm identification, generalized search benchmarks

• Offline Reinforcement Learning with Adaptive Feature Fusion

  • Tieru Wang, Kunbao Wu, Guoshun Nan
  • Key: Decision Transformer, Return-Conditioned Supervised Learning, Offline RL, Trajectory Stitching
  • ExpEnv: D4RL

• Peak-Return Greedy Slicing: Subtrajectory Selection for Transformer-based Offline RL

  • Zhiwei Xu, Miduo Cui, Dapeng Li, Zhihao Liu, Haifeng Zhang, Hangyu Mao, Guoliang Fan, Bin Zhang
  • Key: Transformer-Based Offline RL, Subtrajectory Selection, Return-to-Go, Stitching
  • ExpEnv: offline RL benchmarks

• TrojanTO: Action-Level Backdoor Attacks Against Trajectory Optimization Models

  • Yang Dai, Oubo Ma, Xingxing Liang, Longfei Zhang, Xiaochun Cao, Shouling Ji, Jiaheng Zhang, Jincai Huang, Li Shen
  • Key: Decision Transformer, Backdoor Attacks, Offline RL, Trajectory Optimization
  • ExpEnv: D4RL (DT/TT)

• GAS: Enhancing Reward-Cost Balance of Generative Model-assisted Offline Safe RL

  • Zifan Liu, Xinran Li, Shibo Chen, Jun Zhang
  • Key: Offline Safe RL, Constrained Decision Transformer, Trajectory Stitching, Generative Models
  • ExpEnv: offline safe RL

NeurIPS 2025

• Value-Guided Decision Transformer: A Unified Reinforcement Learning Framework for Online and Offline Settings

  • Hongling Zheng, Li Shen, Yong Luo, Deheng Ye, Shuhan Xu, Bo Du, Jialie Shen, Dacheng Tao
  • Key: Offline RL, Conditional Sequence Modeling, Decision Transformer
  • ExpEnv: Gym, Adroit, Kitchen, AntMaze, Maze2D

• Prompt Tuning Decision Transformers with Structured and Scalable Bandits

  • Finn Rietz, Oleg Smirnov, Sara Karimi, Lele Cao
  • Key: Reinforcement Learning, Decision Transformer, Prompt tuning, Bandits, Multi-task transfer
  • ExpEnv: D4RL

• Adaptable Safe Policy Learning from Multi-task Data with Constraint Prioritized Decision Transformer

  • Ruiqi Xue, Ziqian Zhang, Lihe Li, Cong Guan, Lei Yuan, Yang Yu
  • Key: Reinforcement Learning, Safe Reinforcement Learning, Offline Reinforcement Learning, Multi-task Reinforcement Learning
  • ExpEnv: OSRL

• MOSDT: Self-Distillation-Based Decision Transformer for Multi-Agent Offline Safe Reinforcement Learning

  • Yuchen Xia, Yunjian Xu
  • Key: Multi-Agent Reinforcement Learning, Safe Reinforcement Learning, Offline Reinforcement Learning, Decision Transformer
  • ExpEnv: MOSDB

ICML 2025

• Decision Mixer: Integrating Long-term and Local Dependencies via Dynamic Token Selection for Decision-Making

  • Hongling Zheng, Li Shen, Yong Luo, Deheng Ye, Bo Du, Jialie Shen, Dacheng Tao
  • Key: Offline RL, Conditional Sequence Modeling, Decision Transformer, dynamic token selection, Reinforcement Learning.
  • ExpEnv: D4RL (Gym, Adroit, Kitchen, AntMaze, Maze2D)

• Ad Hoc Teamwork via Offline Goal-Based Decision Transformers

  • Xinzhi Zhang, Hohei Chan, Deheng Ye, Yi Cai, Mengchen Zhao
  • Key: Ad Hoc Teamwork, Offline RL, Multi-agent RL, Decision Transformer, Reinforcement Learning.
  • ExpEnv: Predator-prey (PP), Level-Based Foraging (LBF), Overcooked

• Unveiling Markov heads in Pretrained Language Models for Offline Reinforcement Learning

  • Wenhao Zhao, Qiushui Xu, Linjie Xu, Lei Song, Jinyu Wang, Chunlai Zhou, Jiang Bian
  • Key: Decision Transformer, Offline RL, Pretrained Language Models (PLMs), Markov head, Reinforcement Learning.
  • ExpEnv: MuJoCo, PointMaze

• Behavioral Exploration: Learning to Explore via In-Context Adaptation

  • Andrew Wagenmaker, Zhiyuan Zhou, Sergey Levine
  • Key: in-context learning, exploration, behavior cloning, adaptive agents, Reinforcement Learning.
  • ExpEnv: D4RL (Antmaze, Kitchen), simulated robotics, real-world robotics.

ICLR 2025

• Long-Short Decision Transformer: Bridging Global and Local Dependencies for Generalized Decision-Making
  • Jincheng Wang, Penny Karanasou, Pengyuan Wei, Elia Gatti, Diego Martinez Plasencia, Dimitrios Kanoulas
  • Key: Deep Learning, Reinforcement Learning, Transformer, Decision Transformer, Long-Short Decision Transformer, OfflineRL
  • ExpEnv: D4RL offline RL benchmark, Maze2d, Antmaze

NeurIPS 2024

• Adaptive Q-Aid for Conditional Supervised Learning in Offline Reinforcement Learning

  • Jeonghye Kim, Suyoung Lee, Woojun Kim, Youngchul Sung
  • Keyword: Q-learning, DT
  • ExpEnv: MuJoco, AntMaze, Adroit

• Meta-DT: Offline Meta-RL as Conditional Sequence Modeling with World Model Disentanglement

  • Zhi Wang, Li Zhang, Wenhao Wu, Yuanheng Zhu, Dongbin Zhao, Chunlin Chen
  • Keyword: Offline Meta-Reinforcement Learning, Transformer, World Model Disentanglement
  • ExpEnv: MuJoCo, Meta-World

• Decomposed Prompt Decision Transformer for Efficient Unseen Task Generalization

  • Hongling Zheng, Li Shen, Yong Luo, Tongliang Liu, Jialie Shen, Dacheng Tao
  • Publisher: NeurIPS 2024
  • Key: Offline Reinforcement Learning, Prompt Tuning
  • ExpEnv: MuJoco

IROS 2024

• Steering Decision Transformers via Temporal Difference Learning
  • Hao-Lun Hsu, Alper Kamil Bozkurt, Juncheng Dong, Qitong Gao, Vahid Tarokh, Miroslav Pajic
  • Publisher: IROS 2024
  • Key: Robotics, offline reinforcement learning, sequence modeling
  • ExpEnv: D4RL, Robotic Manipulation, CARLA

ICML 2024

• Generalization to New Sequential Decision Making Tasks with In-Context Learning

  • Sharath Chandra Raparthy, Eric Hambro, Robert Kirk, Mikael Henaff, Roberta Raileanu
  • Publisher: ICML 2024
  • Key: DT, In-Context Learning
  • ExpEnv: MiniHack, Procgen

• HarmoDT: Harmony Multi-Task Decision Transformer for Offline Reinforcement Learning

  • Shengchao Hu, Ziqing Fan, Li Shen, Ya Zhang, Yanfeng Wang, Dacheng Tao
  • Publisher: ICML 2024
  • Key: Multi-task, DT
  • ExpEnv: MetaWorld

• Q-value Regularized Transformer for Offline Reinforcement Learning

  • Shengchao Hu, Ziqing Fan, Chaoqin Huang, Li Shen, Ya Zhang, Yanfeng Wang, Dacheng Tao
  • Publisher: ICML 2024
  • Key: Q-learning, DT
  • ExpEnv: D4RL

• Temporal Logic Specification-Conditioned Decision Transformer for Offline Safe Reinforcement Learning

  • Zijian Guo, Weichao Zhou, Wenchao Li
  • Publisher: ICML 2024
  • Key: Signal Temporal Logic (STL), DT
  • ExpEnv: DSRL

• Think Before You Act: Decision Transformers with Working Memory

  • Jikun Kang, Romain Laroche, Xingdi Yuan, Adam Trischler, Xue Liu, Jie Fu
  • Publisher: ICML 2024
  • Key: Working Memory, DT
  • ExpEnv: MetaWorld, Atari

• In-Context Decision Transformer: Reinforcement Learning via Hierarchical Chain-of-Thought

  • Sili Huang, Jifeng Hu, Hechang Chen, Lichao Sun, Bo Yang
  • Publisher: ICML 2024
  • Key: Hierarchical Structure, DT
  • ExpEnv: D4RL

• Rethinking Decision Transformer via Hierarchical Reinforcement Learning

  • Yi Ma, Jianye Hao, Hebin Liang, Chenjun Xiao,
  • Publisher: ICML 2024
  • Key: DT, Hierarchical Reinforcement Learning
  • ExpEnv: D4RL

ICLR 2024

• Learning Multi-Agent Communication from Graph Modeling Perspective

  • Shengchao Hu, Li Shen, Ya Zhang, Dacheng Tao
  • Publisher: ICLR 2024
  • Key: Communication, Sequence Modeling
  • ExpEnv: SMAC

• Decision ConvFormer: Local Filtering in MetaFormer is Sufficient for Decision Making

  • Jeonghye Kim, Suyoung Lee, Woojun Kim, Youngchul Sung
  • Key: MetaFormer, Decision ConvFormer
  • ExpEnv: D4RL, Atari

• When should we prefer Decision Transformers for Offline Reinforcement Learning?

  • Prajjwal Bhargava, Rohan Chitnis, Alborz Geramifard, Shagun Sodhani, Amy Zhang
  • Key: offline reinforcement learning, sequence modeling, reinforcement learning
  • ExpEnv: D4RL

• Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretraining

  • Licong Lin, Yu Bai, Song Mei
  • Key: transformers, in-context learning, reinforcement learning, learning theory
  • ExpEnv: stochastic linear bandit

• Searching for High-Value Molecules Using Reinforcement Learning and Transformers

  • Raj Ghugare, Santiago Miret, Adriana Hugessen, Mariano Phielipp, Glen Berseth
  • Key: chemistry, reinforcement learning, language models
  • ExpEnv: docking and pytdc tasks

NeurIPS 2023

• SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Tasks

  • Bill Yuchen Lin, Yicheng Fu, Karina Yang, Faeze Brahman, Shiyu Huang, Chandra Bhagavatula, Prithviraj Ammanabrolu, Yejin Choi, Xiang Ren
  • Publisher: NeurIPS 2023
  • Key: Dual-Process Theory
  • ExpEnv: ScienceWorld

• HIQL: Offline Goal-Conditioned RL with Latent States as Actions

  • Seohong Park, Dibya Ghosh, Benjamin Eysenbach, Sergey Levine
  • Publisher: NeurIPS 2023
  • Key: Hierarchical Goal-Conditioned RL, Offline Reinforcement Learning, Value Function Estimation
  • ExpEnv: D4RL

• Structured state space models for in-context reinforcement learning

  • Chris Lu, Yannick Schroecker, Albert Gu, Emilio Parisotto, Jakob Foerster, Satinder Singh, Feryal Behbahani
  • Publisher: NeurIPS 2023
  • Key: state space model, in-context RL
  • ExpEnv: t-maze

• Supervised Pretraining Can Learn In-Context Reinforcement Learning

  • Jonathan N. Lee, Annie Xie, Aldo Pacchiano, Yash Chandak, Chelsea Finn, Ofir Nachum, Emma Brunskill
  • Publisher: NeurIPS 2023
  • Key: Decision Pretrained Transformer, in-context learning
  • ExpEnv: Dark Room

• Is Feedback All You Need? Leveraging Natural Language Feedback in Goal-Conditioned Reinforcement Learning

  • Sabrina McCallum, Max Taylor-Davies, Stefano V. Albrecht, Alessandro Suglia
  • Publisher: NeurIPS 2023 Workshop
  • Key: DT, language feedback
  • ExpEnv: BabyAI

• STEVE-1: A Generative Model for Text-to-Behavior in Minecraft

  • Shalev Lifshitz, Keiran Paster, Harris Chan, Jimmy Ba, Sheila McIlraith
  • Publisher: NeurIPS 2023
  • Key: instruction-tuned Video Pretraining
  • ExpEnv: Minecraft

• Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection

  • Yu Bai, Fan Chen, Huan Wang, Caiming Xiong, Song Mei
  • Publisher: NeurIPS 2023
  • Key: in-context learning, transformers, deep learning theory, learning theory
  • ExpEnv: in-context regression problems

• Elastic Decision Transformer

  • Yueh-Hua Wu, Xiaolong Wang, Masashi Hamaya
  • Publisher: NeurIPS 2023
  • Key: Offline RL, stitch trajectory, Multi-Task
  • ExpEnv: D4RL

• Learning to Modulate pre-trained Models in RL

  • Thomas Schmied, Markus Hofmarcher, Fabian Paischer, Razvan Pascanu, Sepp Hochreiter
  • Publisher: NeurIPS 2023 (Poster)
  • Key: reinforcement learning, multi-task learning, continual learning, fine-tuning
  • ExpEnv: MetaWorld, DMControl

CoRL 2023

• Transformers are Adaptable Task Planners

  • Vidhi Jain, Yixin Lin, Eric Undersander, Yonatan Bisk, Akshara Rai
  • Publisher: CoRL 2023
  • Key: Task Planning, Prompt, Control, Generalization
  • Code: official
  • ExpEnv: Dishwasher Loading

• Q-Transformer

  • Yevgen Chebotar, Quan Vuong, Alex Irpan, Karol Hausman, Fei Xia, Yao Lu, Aviral Kumar, Tianhe Yu, Alexander Herzog, Karl Pertsch, Keerthana Gopalakrishnan, Julian Ibarz, Ofir Nachum, Sumedh Sontakke, Grecia Salazar, Huong T Tran, Jodilyn Peralta, Clayton Tan, Deeksha Manjunath, Jaspiar Singht, Brianna Zitkovich, Tomas Jackson, Kanishka Rao, Chelsea Finn, Sergey Levine
  • Publisher: CoRL 2023
  • Key: Reinforcement Learning, Offline RL, Transformers, Q-Learning, Robotic Manipulation
  • Code: Unofficial
  • ExpEnv: None

IROS 2023

• Hierarchical Decision Transformer

  • André Correia, Luís A. Alexandre
  • Publisher: IROS 2023
  • Key: Hierarchical Learning, Imitation Learning
  • ExpEnv: MuJoco, D4RL, RoboMimic, Maze 2D

• PACT: Perception-Action Causal Transformer for Autoregressive Robotics Pre-Training

  • Rogerio Bonatti, Sai Vemprala, Shuang Ma, Felipe Frujeri, Shuhang Chen, Ashish Kapoor
  • Publisher: IROS 2023
  • Key: Robotics, Pretrain, Multitask, Representation
  • ExpEnv: MuSHR car, Habitat

ICML 2023

• Constrained Decision Transformer for Offline Safe Reinforcement Learning

  • Zuxin Liu, Zijian Guo, Yihang Yao, Zhepeng Cen, Wenhao Yu, Tingnan Zhang, Ding Zhao
  • Publisher: ICML 2023
  • Key: Offline Safe RL, DT
  • ExpEnv: Bullet-Safety-Gym

• Q-learning Decision Transformer: Leveraging Dynamic Programming for Conditional Sequence Modelling in Offline RL

  • Taku Yamagata, Ahmed Khalil, Raul Santos-Rodriguez
  • Publisher: ICML 2023
  • Key: Q-Learning
  • ExpEnv: D4RL

ICRA 2023

• LATTE: LAnguage Trajectory TransformEr
  • Arthur Bucker, Luis Figueredo, Sami Haddadin, Ashish Kapoor, Shuang Ma, Sai Vemprala, Rogerio Bonatti
  • Publisher: ICRA 2023
  • Key: MultiModal, Robotics
  • Code: official, official
  • ExpEnv: CoppeliaSim

ICLR 2023

• In-context Reinforcement Learning with Algorithm Distillation

  • Michael Laskin, Luyu Wang, Junhyuk Oh, Emilio Parisotto, Stephen Spencer, Richie Steigerwald, DJ Strouse, Steven Stenberg Hansen, Angelos Filos, Ethan Brooks, maxime gazeau, Himanshu Sahni, Satinder Singh, Volodymyr Mnih
  • Publisher: ICLR 2023
  • Key: Reinforcement Learning, Transformers, Learning to Learn, Large Language Models
  • ExpEnv: Adversarial Bandit, Dark Room, Dark Key-to-Door, DMLab Watermaze

• EDGI: Equivariant Diffusion for Planning with Embodied Agents

  • Johann Brehmer, Joey Bose, Pim de Haan, Taco Cohen
  • Publisher: ICLR 2023 Reincarnating RL workshop
  • Key: rich geometric structure, equivariant, conditional generative modeling, representation
  • ExpEnv: None

• Learning to Modulate pre-trained Models in RL

  • Thomas Schmied, Markus Hofmarcher, Fabian Paischer, Razvan Pascanu, Sepp Hochreiter
  • Publisher: ICLR 2023 Reincarnating RL workshop
  • Key: reinforcement learning, multi-task learning, continual learning, fine-tuning
  • ExpEnv: MetaWorld, DMControl

• DeFog: Decision Transformer under Random Frame Dropping

  • Kaizhe Hu*, Ray Chen Zheng*, Yang Gao, Huazhe Xu
  • Publisher: ICLR 2023
  • Key: Offline RL, POMDP, Frame-Dropping, Practical Application
  • Code: official
  • ExpEnv: MuJoco, D4RL, Atari

NeurIPS 2022

• When does return-conditioned supervised learning work for offline reinforcement learning?

  • David Brandfonbrener, Alberto Bietti, Jacob Buckman, Romain Laroche, Joan Bruna
  • Publisher: NeurIPS 2022
  • Key: Theoretical analysis
  • ExpEnv: MuJoco, D4RL

• You Can't Count on Luck: Why Decision Transformers Fail in Stochastic Environments

  • Keiran Paster, Sheila McIlraith, Jimmy Ba
  • Publisher: NeurIPS 2022
  • Key: Stochastic Environments
  • ExpEnv: Gambling, Connect Four, 2048

• Multi-Agent Reinforcement Learning is a Sequence Modeling Problem

  • Muning Wen, Jakub Grudzien Kuba, Runji Lin, Weinan Zhang, Ying Wen, Jun Wang, Yaodong Yang
  • Publisher: NeurIPS 2022
  • Key: Multi-Agent RL
  • ExpEnv: SMAC, MA MuJoco

• Bootstrapped Transformer for Offline Reinforcement Learning

  • Kerong Wang, Hanye Zhao, Xufang Luo, Kan Ren, Weinan Zhang, Dongsheng Li
  • Publisher: NeurIPS 2022
  • Key: Generation model
  • Code: official
  • ExpEnv: MuJoco, D4RL, Adroit

• Multi-Game Decision Transformers

  • Kuang-Huei Lee, Ofir Nachum, Mengjiao Yang, Lisa Lee, Daniel Freeman, Winnie Xu, Sergio Guadarrama, Ian Fischer, Eric Jang, Henryk Michalewski, Igor Mordatch
  • Publisher: NeurIPS 2022
  • Key: Multi-Task, Finetuning
  • Code: official
  • ExpEnv: Atari, REM

• Decision making as language generation

  • Roland Memisevic, Sunny Panchal, Mingu Lee
  • Publisher: NeurIPS 2022 Workshop FMDM
  • Key: Generation
  • ExpEnv: Traversals (Toy experiment)

CoRL 2022

• Offline Reinforcement Learning for Customizable Visual Navigation

  • Dhruv Shah, Arjun Bhorkar, Hrishit Leen, Ilya Kostrikov, Nicholas Rhinehart, Sergey Levine
  • Publisher: CoRL 2022 (Oral)
  • Key: Visual Navigation
  • ExpEnv: RECON

• Instruction-driven history-aware policies for robotic manipulations

  • Pierre-Louis Guhur, Shizhe Chen, Ricardo Garcia, Makarand Tapaswi, Ivan Laptev, Cordelia Schmid
  • Publisher: CoRL 2022 (Oral)
  • Key: Robotics, Language Instruction
  • Code: official
  • ExpEnv: RLBench

• Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation

  • Mohit Shridhar, Lucas Manuelli, Dieter Fox
  • Publisher: CoRL 2022
  • Key: Robotics, Language Grounding, Behavior Cloning
  • Code: official
  • ExpEnv: RLBench

ICML 2022

• Online Decision Transformer

  • Qinqing Zheng, Amy Zhang, Aditya Grover
  • Publisher: ICML 2022 (Oral)
  • Key: Online finetuning, Max-entropy, Exploration
  • Code: unofficial
  • ExpEnv: MuJoco, D4RL

• Prompting Decision Transformer for Few-Shot Policy Generalization

  • Mengdi Xu, Yikang Shen, Shun Zhang, Yuchen Lu, Ding Zhao, Joshua B. Tenenbaum, Chuang Gan
  • Publisher: ICML 2022 (Poster)
  • Key: Prompt, Few-shot, Generalization
  • Code: official (released soon)
  • ExpEnv: DMC

• Addressing Optimism Bias in Sequence Modeling for Reinforcement Learning

  • Adam R Villaflor, Zhe Huang, Swapnil Pande, John M Dolan, Jeff Schneider
  • Publisher: ICML 2022 (Poster)
  • Key: World model
  • Code: official (released soon)
  • ExpEnv: CARLA

• AnyMorph: Learning Transferable Polices By Inferring Agent Morphology

  • Brandon Trabucco, Mariano Phielipp, Glen Berseth
  • Publisher: ICML 2022 (Poster)
  • Key: Morphology, Transfer Learning, Zero Shot
  • ExpEnv: Modular-RL

AAAI 2022

• Dreaming with Transformers
  • Catherine Zeng, Jordan Docter, Alexander Amini, Igor Gilitschenski, Ramin Hasani, Daniela Rus
  • Publisher: AAAI 2022 (RLG Workshop)
  • Key: Dreamer, World Model
  • ExpEnv: Deepmind Lab, VISTA

ICLR 2022

• Learning Transferable Policies By Inferring Agent Morphology

  • Brandon Trabucco, Mariano Phielipp, Glen Berseth
  • Publisher: ICLR 2022 (GPL Workshop Poster)
  • Key: Morphology, Transfer Learning, Zero Shot
  • ExpEnv: Modular-RL

• Silver-Bullet-3D at ManiSkill 2021: Learning-from-Demonstrations and Heuristic Rule-based Methods for Object Manipulation

  • Yingwei Pan, Yehao Li, Yiheng Zhang, Qi Cai, Fuchen Long, Zhaofan Qiu, Ting Yao, Tao Mei
  • Publisher: ICLR 2022 (GPL Workshop Poster)
  • Key: Object Manipulation
  • Code: official
  • ExpEnv: ManiSkill

• Generalized Decision Transformer for Offline Hindsight Information Matching

  • Hiroki Furuta, Yutaka Matsuo, Shixiang Shane Gu
  • Publisher: ICLR 2021 (Spotlight)
  • Key: HIM, SMM
  • Code: official
  • ExpEnv: MuJoco, D4RL

NeurIPS 2021

• Decision Transformer: Reinforcement Learning via Sequence Modeling

  • Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Michael Laskin, Pieter Abbeel, Aravind Srinivas, Igor Mordatch
  • Publisher: NeurIPS 2021 (Poster)
  • Key: Conditional sequence modeling
  • Code: official, DI-engine
  • ExpEnv: MuJoco, D4RL, Atari

• Offline Reinforcement Learning as One Big Sequence Modeling Problem

  • Michael Janner, Qiyang Li, Sergey Levine
  • Publisher: NeurIPS 2021 (Spotlight)
  • Key: Conditional sequence modeling, Discretization
  • Code: official
  • ExpEnv: MuJoco, D4RL

• TransDreamer: Reinforcement Learning with Transformer World Models

  • Chang Chen, Yi-Fu Wu, Jaesik Yoon, Sungjin Ahn
  • Publisher: NeurIPS 2021 (Deep RL Workshop)
  • Key: Dreamer, World Model
  • ExpEnv: Hidden Order Discovery, DMC, Atari

ICML 2021

• Reinforcement learning as one big sequence modeling problem
  • Michael Janner, Qiyang Li, Sergey Levine
  • Publisher: ICML workshop
  • Key: Conditional sequence modeling, Discretization
  • Code: official
  • ExpEnv: MuJoco, D4RL

Contributing

Our purpose is to make this repo even better. If you are interested in contributing, please refer to HERE for instructions in contribution.

License

Awesome Decision Transformer is released under the Apache 2.0 license.