Privileged Feature

Privileged information, data available during training but not at test time, is used to improve model performance in various machine learning tasks. Current research focuses on leveraging this information through techniques like knowledge distillation and constrained reinforcement learning, often employing deep neural networks to learn effective representations from both privileged and non-privileged features. This approach shows promise in improving efficiency and performance in diverse applications, including robotics, natural language processing, and autonomous driving, by enabling the transfer of knowledge learned with extra information to models operating under real-world constraints. However, limitations exist, particularly concerning the potential for overfitting to privileged information and the need for careful consideration of bias-variance trade-offs, especially in sensitive domains.

Papers

September 20, 2024

SoloParkour: Constrained Reinforcement Learning for Visual Locomotion from Privileged Experience
Elliot Chane-Sane, Joseph Amigo, Thomas Flayols, Ludovic Righetti, Nicolas Mansard
Legged Robot Constrained Reinforcement Learning Parkour Skill Privileged Feature Visual Locomotion

June 24, 2024

The Privileged Students: On the Value of Initialization in Multilingual Knowledge Distillation
Haryo Akbarianto Wibowo, Thamar Solorio, Alham Fikri Aji
Knowledge Distillation NLP Task Net Present Value New Initialization Multilingual Scenario Multilingual Capability Multilingual Knowledge Privileged Feature

September 18, 2023

Privileged to Predicted: Towards Sensorimotor Reinforcement Learning for Urban Driving
Ege Onat Özsüer, Barış Akgün, Fatma Güney
Reinforcement Learning Autonomous Driving Reinforcement Learning Algorithm Privileged Information Urban Driving Multiple Prediction Privileged Feature Sensorimotor Policy

July 24, 2023

On Privileged and Convergent Bases in Neural Network Representations
Davis Brown, Nikhil Vyas, Yamini Bansal
Neural Network Neural Representation Individual Neuron Neural Network Representation Privileged Feature

March 5, 2023

On the Capacity Limits of Privileged ERM
Michal Sharoni, Sivan Sabato
Supervised Learning Privileged Information Capacity Constraint Privileged Feature

February 17, 2023

The Unbearable Weight of Massive Privilege: Revisiting Bias-Variance Trade-Offs in the Context of Fair Prediction
Falaah Arif Khan, Julia Stoyanovich
Context Information Gaussian Mixture Model Bias Variance FAir Classifier Conditional Model Bias Variance Decomposition Fair Prediction Weight Monitoring Privileged Feature

September 19, 2022

Toward Understanding Privileged Features Distillation in Learning-to-Rank
Shuo Yang, Sujay Sanghavi, Holakou Rahmanian, Jan Bakus, S. V. N. Vishwanathan
Learning to Rank Privileged Knowledge Distillation Teacher Selection Privileged Feature

Privileged Feature

Papers

SoloParkour: Constrained Reinforcement Learning for Visual Locomotion from Privileged Experience

The Privileged Students: On the Value of Initialization in Multilingual Knowledge Distillation

Privileged to Predicted: Towards Sensorimotor Reinforcement Learning for Urban Driving

On Privileged and Convergent Bases in Neural Network Representations

On the Capacity Limits of Privileged ERM

The Unbearable Weight of Massive Privilege: Revisiting Bias-Variance Trade-Offs in the Context of Fair Prediction

Toward Understanding Privileged Features Distillation in Learning-to-Rank