Downstream Policy Learning

Downstream policy learning focuses on effectively leveraging pre-trained models or datasets to improve the performance and efficiency of subsequent machine learning tasks, particularly in reinforcement learning and other sequential decision-making problems. Current research emphasizes developing robust methods for cross-domain adaptation, mitigating vulnerabilities in pre-trained encoders used in downstream services, and improving sample efficiency through techniques like synthetic data generation and self-supervised learning. These advancements are significant because they address key challenges in applying machine learning to complex real-world problems, such as robotics and financial prediction, by enabling more efficient and reliable learning from limited or noisy data.

Papers

September 13, 2024

xTED: Cross-Domain Policy Adaptation via Diffusion-Based Trajectory Editing
Haoyi Niu, Qimao Chen, Tenglong Liu, Jianxiong Li, Guyue Zhou, Yi Zhang, Jianming Hu, Xianyuan Zhan
Trajectory Refinement Downstream Policy Learning Cross Domain Policy Transfer

August 5, 2024

Pre-trained Encoder Inference: Revealing Upstream Encoders In Downstream Machine Learning Services
Shaopeng Fu, Xuexue Sun, Ke Qing, Tianhang Zheng, Di Wang
Machine Learning Adversarial Attack Pre Trained Downstream Policy Learning

June 13, 2024

A Dual Approach to Imitation Learning from Observations with Offline Datasets
Harshit Sikchi, Caleb Chuck, Amy Zhang, Scott Niekum
Imitation Learning Evolving Observation Offline Data Expert Level Performance Downstream Policy Learning Dual Approach Inverse Dynamic Imperfect Demonstration

May 31, 2024

Mamba State-Space Models Are Lyapunov-Stable Learners
John T. Halloran, Manbir Gulati, Paul F. Roysdon
Large Language Model Fine Tuning Downstream Policy Learning

May 20, 2024

Learning Future Representation with Synthetic Observations for Sample-efficient Reinforcement Learning
Xin Liu, Yaran Chen, Dongbin Zhao
Synthetic Data Visual Reinforcement Learning Sample Efficient Reinforcement Learning Downstream Policy Learning Self Supervised Reinforcement Learning Future Latent Auxiliary Function

October 21, 2023

Towards a General Framework for Continual Learning with Pre-training
Liyuan Wang, Jingyi Xie, Xingxing Zhang, Hang Su, Jun Zhu
Continual LEArning General Framework Downstream Policy Learning Within Task Prediction

October 11, 2023

Hierarchical Decomposition of Prompt-Based Continual Learning: Rethinking Obscured Sub-optimality
Liyuan Wang, Jingyi Xie, Xingxing Zhang, Mingyi Huang, Hang Su, Jun Zhu
Continual LEArning Self Supervised Pre Training Downstream Policy Learning Prompt Based Continual Learning Hierarchical Decomposition

October 6, 2023

Copy Suppression: Comprehensively Understanding an Attention Head
Callum McDougall, Arthur Conmy, Cody Rushing, Thomas McGrath, Neel Nanda
Attention Head Attention Operation Copy Suppression Downstream Policy Learning Self Repair

April 10, 2023

For Pre-Trained Vision Models in Motor Control, Not All Policy Learning Methods are Created Equal
Yingdong Hu, Renhao Wang, Li Erran Li, Yang Gao
Reinforcement Learning Pre Trained Model Pre Trained Policy Learning Method Downstream Policy Learning Motion Skill

February 11, 2023

Cross-domain Random Pre-training with Prototypes for Reinforcement Learning
Xin Liu, Yaran Chen, Haoran Li, Boyu Li, Dongbin Zhao
Reinforcement Learning Visual Prototype Domain Randomization Downstream Policy Learning Visual Control Cross Domain Reinforcement Learning

September 12, 2022

An Evaluation of Low Overhead Time Series Preprocessing Techniques for Downstream Machine Learning
Matthew L. Weiss, Joseph McDonald, David Bestor, Charles Yee, Daniel Edelman, Michael Jones, Andrew Prout, Andrew Bowne, Lindsey McEvoy, Vijay Gadepally, Siddharth Samsi
Global Evaluation Downstream Policy Learning Cloud Workload Multi Channel Time Series

May 30, 2022

Meta Representation Learning with Contextual Linear Bandits
Leonardo Cella, Karim Lounici, Massimiliano Pontil
Meta Learning Linear Bandit Linear Contextual Bandit Meta Representation Downstream Policy Learning Minimax Optimal Algorithm

May 17, 2022

Self-Supervised Learning of Multi-Object Keypoints for Robotic Manipulation
Jan Ole von Hartz, Eugenio Chisari, Tim Welschehold, Abhinav Valada
Self Supervised Learning Robotic Manipulation Manipulation Task Arbitrary Keypoints Downstream Policy Learning Policy Learning Method Sample Efficient Policy

April 30, 2022

Continual Learning with Foundation Models: An Empirical Study of Latent Replay
Oleksiy Ostapenko, Timothee Lesort, Pau Rodríguez, Md Rifat Arefin, Arthur Douillard, Irina Rish, Laurent Charlin
Continual LEArning Foundation Model Pre Trained Empirical Study Pre Trained Convolutional Neural Network Large Scale Pre Training Downstream Policy Learning Latent Replay

March 30, 2022

Marginalized Operators for Off-policy Reinforcement Learning
Yunhao Tang, Mark Rowland, Rémi Munos, Michal Valko
Reinforcement Learning Policy Evaluation Policy Reinforcement Learning Downstream Policy Learning Stochastic Operator

December 19, 2021

Denoised Labels for Financial Time-Series Data via Self-Supervised Learning
Yanqing Ma, Carmine Ventre, Maria Polukarov
Self Supervised Learning Label Information Financial Time Series Label Denoising Downstream Policy Learning