Return Conditioned Supervised Learning

Return-conditioned supervised learning (RCSL) focuses on training models that predict actions or outputs conditioned on desired future outcomes, such as high rewards or specific target values. Current research explores various model architectures, including diffusion models and decision transformers, and investigates techniques to improve robustness and efficiency, addressing challenges like hyperparameter sensitivity and the impact of environmental stochasticity. This approach holds significant promise for advancing offline reinforcement learning, improving the efficiency of conditional generation tasks in diverse fields, and enabling more controllable and aligned AI systems.

Papers

October 4, 2024

Training Over a Distribution of Hyperparameters for Enhanced Performance and Adaptability on Imbalanced Classification
Kelsey Lieberman, Swarna Kamlam Ravindran, Shuai Yuan, Carlo Tomasi
Classification Code Loss Function Adaptive Importance Binary Classification Hyperparameter Tuning Related Hyperparameters Performance Improvement Training Model Product Distribution Return Conditioned Supervised Learning

July 2, 2024

No Training, No Problem: Rethinking Classifier-Free Guidance for Diffusion Models
Seyedmorteza Sadat, Manuel Kansy, Otmar Hilliges, Romann M. Weber
Diffusion Model Training Data Conditional Diffusion Model Right Problem Classifier Free Guidance Return Conditioned Supervised Learning

June 18, 2024

Extracting Training Data from Unconditional Diffusion Models
Yunhao Chen, Xingjun Ma, Difan Zou, Yu-Gang Jiang
Training Data Generative Artificial Intelligence Probabilistic Model Conditional Diffusion Model Return Conditioned Supervised Learning Unconditional Diffusion Model

June 15, 2024

CoLoR-Filter: Conditional Loss Reduction Filtering for Targeted Language Model Pre-training
David Brandfonbrener, Hanlin Zhang, Andreas Kirsch, Jonathan Richard Schwarz, Sham Kakade
Language Model Data Selection Language Modeling Task Return Conditioned Supervised Learning Language Model Pre Training Color Filter

June 3, 2024

DEFT: Efficient Fine-Tuning of Diffusion Models by Learning the Generalised $h$-transform
Alexander Denker, Francisco Vargas, Shreyas Padhy, Kieran Didi, Simon Mathis, Vincent Dutordoir, Riccardo Barbano, Emile Mathieu, Urszula Julia Komorowska, Pietro Lio
Strong Generalization Fine Tuning Image Reconstruction Conditional Diffusion Model Conditional Sampling Return Conditioned Supervised Learning Unconditional Diffusion Model

April 22, 2024

Ungeneralizable Examples
Jingwen Ye, Xinchao Wang
Return Conditioned Supervised Learning Unlearnable Datasets Positive Example

April 7, 2024

Regularized Conditional Diffusion Model for Multi-Task Preference Alignment
Xudong Yu, Chenjia Bai, Haoran He, Changhong Wang, Xuelong Li
Conditional Diffusion Model Sequential Decision Preference Alignment Return Conditioned Supervised Learning Preference Representation Task Preference

February 8, 2024

Optimizing for ROC Curves on Class-Imbalanced Data by Training over a Family of Loss Functions
Kelsey Lieberman, Shuai Yuan, Swarna Kamlam Ravindran, Carlo Tomasi
Training Data Loss Function Binary Classification Efficient Optimization Imbalanced Classification ROC Curve Return Conditioned Supervised Learning Class Imbalanced Data

February 3, 2024

Adaptive $Q$-Aid for Conditional Supervised Learning in Offline Reinforcement Learning
Jeonghye Kim, Suyoung Lee, Woojun Kim, Youngchul Sung
Offline Reinforcement Learning Return Conditioned Supervised Learning Value Based

December 21, 2023

Critic-Guided Decision Transformer for Offline Reinforcement Learning
Yuanfu Wang, Chao Yang, Ying Wen, Yu Liu, Yu Qiao
Offline Reinforcement Learning Value Function Decision Transformer Critic Model Return Conditioned Supervised Learning

May 19, 2023

Conditional Online Learning for Keyword Spotting
Michel Meneses, Bruno Iwami
Pre Trained Model Continual Learning Keyword Spotting Return Conditioned Supervised Learning Training Distribution Naive Adaptation

April 17, 2023

Conditional Generation of Audio from Video via Foley Analogies
Yuexi Du, Ziyang Chen, Justin Salamon, Bryan Russell, Andrew Owens
Source Video Audio Driven Conditional Generation Audio Effect Return Conditioned Supervised Learning Visual Analogy Silent Video

March 11, 2023

Lossless Point Cloud Geometry and Attribute Compression Using a Learned Conditional Probability Model
Dat Thanh Nguyen, Andre Kaup
Point Cloud Point Cloud Learning Return Conditioned Supervised Learning Attribute Compression

February 20, 2023

DINOISER: Diffused Conditional Sequence Learning by Manipulating Noises
Jiasheng Ye, Zaixiang Zheng, Yu Bao, Lihua Qian, Mingxuan Wang
Generative Model Sequential Data Return Conditioned Supervised Learning Audio Manipulation Sequence Diffusion

February 16, 2023

Pretraining Language Models with Human Preferences
Tomasz Korbak, Kejian Shi, Angelica Chen, Rasika Bhalerao, Christopher L. Buckley, Jason Phang, Samuel R. Bowman, Ethan Perez
Language Model Human Feedback Task Specific Human Preference Return Conditioned Supervised Learning

December 1, 2022

Why Are Conditional Generative Models Better Than Unconditional Ones?
Fan Bao, Chongxuan Li, Jiacheng Sun, Jun Zhu
Diffusion Model Conditional Generative Score Based Diffusion Model Return Conditioned Supervised Learning Unconditional Model

November 29, 2022

Transfer Entropy Bottleneck: Learning Sequence to Sequence Information Transfer
Damjan Kalajdzievski, Ximeng Mao, Pascal Fortier-Poisson, Guillaume Lajoie, Blake Richards
Information Bottleneck Sequence of Sequence Link Stream Return Conditioned Supervised Learning Information Transfer

November 26, 2022

CFNet: Conditional Filter Learning with Dynamic Noise Estimation for Real Image Denoising
Yifan Zuo, Jiacheng Xie, Yuming Fang, Yan Huang, Wenhui Jiang
Noise Estimation Real World Image Denoising Return Conditioned Supervised Learning Iterative Denoising Non Blind Denoising Camera Pipeline

October 24, 2022

Dichotomy of Control: Separating What You Can Control from What You Cannot
Mengjiao Yang, Dale Schuurmans, Pieter Abbeel, Ofir Nachum
Offline Reinforcement Learning External Control Multi Scenario Stochastic Reward Return Conditioned Supervised Learning Market Return

June 2, 2022

When does return-conditioned supervised learning work for offline reinforcement learning?
David Brandfonbrener, Alberto Bietti, Jacob Buckman, Romain Laroche, Joan Bruna
Offline Reinforcement Learning Optimal Policy Dynamic Programming Return Conditioned Supervised Learning Supervised Learning Problem