Active Preference Learning

Follow this topic Ask about this topic

Latest papers

16 papers5 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Asking Smarter Questions: Targeted Active Preference Learning for Underspecified Rewards
Oct 15, 2026Helena Merker, Nick Walker, Andreea BobuActive Preference Learning Preference Learning
Massachusetts Institute of Technology
SPEAR: Selective Preference Elicitation with Adaptive Rewards for VLM-Guided Robotic Manipulation
Oct 15, 2026Zewen Wu, Ruizhe Liu, Pei Zhou +2Active Preference Learning Preference Learning
HKU Musketeers Foundation Institute of Data Science, The University of Hong Kong, Hong Kong SAR, China. · Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong SAR, China.
Large Language Model Orchestration under Heterogeneous Preferences via Explicit Persona Inference
Oct 6, 2026Shuqing Shi, Ziyan Wang, Milind Tambe +1Multi-Agent Coordination Active Preference Learning
King’s College London, London, United Kingdom · Harvard University, Cambridge, MA, USA
Targeted Active Learning for Preference-Based Treatment Effects on Multivariate Outcomes
Oct 2, 2026Lola Giordani, Mathieu Even, Chloé Geoffroy +3Active Preference Learning Individualized Treatment Effect Estimation
Theremia, Paris · Paris Brain Institute (ICM), Sorbonne Université, Inserm, CNRS, AP-HP · Inria, Inserm, Université de Montpellier, France +1
Optimal Design for Active Preference Learning with Biased LLM Judges
Sep 30, 2026Zhongman Du, Huiming Zhang, Haodong Zhu +1Pairwise Preference Learning LLM Alignment
Beihang University · Zhongguancun Academy · Hangzhou Innovation Institute of Beihang University
Subspace Inference Enables Efficient Active Reward Learning from Preferences
Sep 3, 2026Yutai Zhou, Erdem BıyıkBayesian RL Active Preference Learning
Thomas Lord Department of Computer Science University of Southern California, Los Angeles, USA
Propose to Learn, Learn to Propose: Evaluability-Aware Assistance under Bounded Rationality
Sep 2, 2026Yifan Zhu, Sammie Katt, Samuel KaskiActive Preference Learning Human-AI Interaction
1ELLIS Institute Finland · Department of Computer Science, Aalto University, Finland · Department of Computer Science, University of Manchester, United Kingdom
PAPA: Online Personalized Active Preference Alignment
Jul 1, 2026Anindya Sarkar, Nasik Muhammad Nafi, Isaac Lyngaas +2Diffusion Model Alignment Active Preference Learning
Washington University in St. Louis · Oak Ridge National Laboratory
From Validator Selection to Portfolio Collection Optimization in Proof-of-Stake Blockchains
Jun 6, 2026Jonas Gehrlein, Grzegorz Miebs, Matteo Brunelli +2Active Preference Learning Portfolio Optimization
Parity Technologies AG, Zug, Switzerland · Institute of Computing Science, Poznan University of Technology, Poznań, Poland · Department of Industrial Engineering, University of Trento, Italy
Active Query Synthesis for Preference Learning
May 25, 2026Namrata Nadagouda, Nauman Ahad, Maegan Tucker +1Pairwise Preference Learning Active Learning
Georgia Institute of Technology · Gauss Labs
Vibrotactile Preference Learning: Uncertainty-Aware Preference Learning for Personalized Vibration Feedback
Apr 22, 2026Rongtao Zhang, Xin Zhu, Masoume Pourebadi Khotbehsara +3Pairwise Preference Learning Active Preference Learning
Department of Computer Science University of Southern California Los Angeles, California, USA
Random Is Hard to Beat: Active Selection in online DPO with Modern LLMs
Apr 3, 2026Giyeong Oh, Junghyun Lee, Jaehyun Park +3Active Preference Learning Language Model Post-Training
Seoul National University · Ewha Womans University · KAIST +1
Active Preference Learning over Latent Preference Archetypes for Many-Objective Bayesian Optimization
Mar 30, 2026Manisha Dubey, Sebastiaan De Peuter, Wanrong Wang +1Active Preference Learning Bayesian Optimization
Department of Computer Science, University of Manchester, UK · Centre for AI in Assistive Autonomy, University of Edinburgh, Scotland, UK · Department of Computer Science, Aalto University, Espoo, Finland +2
Improving through Interaction: Searching Behavioral Representation Spaces with CMA-ES-IG
Mar 9, 2026Nathaniel Dennler, Zhonghao Shi, Yiran Tao +3Active Preference Learning Preference Learning
CSAIL/AeroAstro, Massachusetts Institute of Technology. · Department of Computer Science, University of Southern California.
CRED: Counterfactual Reasoning and Environment Design for Active Preference Learning
Mar 9, 2026Yi-Shiuan Tung, Gyanig Kumar, Wei Jiang +2Active Preference Learning Preference Learning
Department of Computer Science, University of Colorado Boulder
Towards Optimality in Active Preference Learning
Feb 2, 2026Yao Zhao, Kwang-Sung JunActive Preference Learning Active Learning
Department of Computer Science, University of Arizona, Tucson, USA.

Oct 15, 2026J/K move · Enter open · S save

Asking Smarter Questions: Targeted Active Preference Learning for Underspecified Rewards

Helena Merker, Nick Walker, Andreea Bobu

Massachusetts Institute of Technology

Active Preference Learning Preference Learning Robot Learning Preference Elicitation Reward Design for RL

Open paper AskPDF arXiv

Reward learning typically assumes the human's input covers everything that matters for the task. In practice, humans can only attend to so much at once, so their teaching naturally captures some features---or aspects of the task---better than others. Prior work has shown how the robot can identify which features were poorly supervised directly from its teaching data. But the current solution is to ask the human to try again, putting the burden on someone who already struggled to convey those features. We propose shifting this burden to the robot. Our key insight is that the same information that tells the robot what it doesn't know can help it ask better questions. Preference queries offer a low-effort channel for resolving this uncertainty, but only if the robot knows where to direct them and the human knows what to attend to when answering. We act on this insight in two ways: we build an informed prior that targets preference queries at the features that need resolving, and we accompany each query with an explanation that focuses the human's attention on exactly those features. In simulation, the informed prior beats the standard active preference learning baseline by up to

30.8\%

in the low-query regime, where reducing human effort matters most. In a user study, the informed prior and explanations each significantly improve reward recovery and amplify each other.