Preference Learning

Follow this topic Ask about this topic

Recent momentum

-33%

4 papers in the last 28 days · 0.1% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-14

2 new papers

A weekly snapshot of new work published in Preference Learning.

Period ending 2026-09-07

1 new paper

A weekly snapshot of new work published in Preference Learning.

54 papers

Latest in Preference Learning

Open your feed →

Preference Optimization with LALM Feedback for Continuous Autoregressive Non-Verbal Vocalization Generation
Sep 11, 2026Jingbin Hu, Qirui Zhan, Yuang Cao +7Preference Learning Natural Language Feedback
You Can't Prefer Emotions You Don't Sample: Intensity Undershoot in DPO-Tuned LLMs
Sep 7, 2026Hyunwoo Kim, Usama KhalidValence-Arousal Estimation Emotion
Subspace Inference Enables Efficient Active Reward Learning from Preferences
Sep 3, 2026Yutai Zhou, Erdem BıyıkPreference Learning Bayesian Neural Networks
PLC-DPO: Posterior Label Correction in Noisy and Ambiguous Preference Optimization
Aug 31, 2026Boryeong Cho, Sumyeong Ahn, Se-Young YunRobust Preference Optimization Preference Learning
Interoceptive Attention as Dynamic Homeostatic Prioritization in a Foraging Agent
Aug 4, 2026St John Grimbly, Nicolas Kuske, Evert A. Boonstra +7Preference Learning Perception-Action Loop
ProPRL: Property-Aware Prerequisite Relation Learning in Educational Knowledge Graphs
Aug 4, 2026Xinghe Cheng, Jiapu Wang, Chaobo He +2Preference Learning Knowledge Graphs
An analysis of machine learning approaches for enhancing decision-making in complex discrete choice tasks
Jul 30, 2026Sheng Lun Christine Cao, Destenie Nock, Alex DavisDiscrete Choice Models Preference Learning
Synchronizing Beliefs with Second-Order Theory-of-Mind in Human-Autonomy Teams (Extended Version)
Jul 29, 2026Jack Mirenzi, Henny AdmoniPreference Learning Teacher
TELLER: Dual-Path Iterative Preference Optimization for Table Entity Linking
Jul 29, 2026Yixin Peng, Kehao Li, Stefan DeckerMulti-Entity Temporal Link Prediction
Learning Dynamic User Personas from Implicit Interaction Streams via Iterative Refinement
Jul 29, 2026Haifeng WuLarge Language Model Personalization Personas
Hard Rules, Soft Preferences: Bridging Reasoning, Learning, and Optimization for Personalized Packing Checklist Generation
Jul 17, 2026Himel Dev, Madhusudan Basak, Tanmoy Sen +2Tour Pairwise Preferences
Step-Level Preference Learning for Generative Agents in Social Simulations
Jul 16, 2026Wenchang Gao, Pingyue Sheng, Lanlan Qiu +7Social Simulations Preference Learning
Internal Pluralism and the Limits of Pairwise Comparisons
Jul 2, 2026Bailey Flanigan, Michelle SiPluralistic Alignment Preference Learning
SPLC: Social Preference Learning for Crowd Robot Navigation
Jul 2, 2026Zixuan Chen, Hao Fu, Haiwen Hu +1Robot Navigation Human-Robot Collaboration
Scaling Trends for Lie Detector Oversight in Preference Learning
Jul 2, 2026Oskar J. Hollinsworth, Ann-Kathrin Dombrowski, Sam Adam-Day +2Deception Preference Learning
Freeform Preference Learning for Robotic Manipulation
Jun 30, 2026Marcel Torne, Anubha Mahajan, Abhijnya Bhat +1Robot Policies Robotic Manipulation
MAPL: Multi-Objective Preference Learning for Robot Locomotion
Jun 24, 2026Xiyue Chen, Muhan Lin, Shuyang Shi +1Perceptive Locomotion Policies Multi-Objective Reinforcement Learning
Towards Spec Learning: Inference-Time Alignment from Preference Pairs
Jun 22, 2026Dhriti Krishnan, Tejas Goyal, Jaromir SavelkaPreference Learning Preference Pairs
Improving Code-Switching ASR with Code-Mixing Guided Synthetic Speech
Jun 14, 2026Yue Heng Yeo, Haoyang Li, Yizhou Peng +6Code-Switching Automatic Speech Recognition
S $^2$ COPE: Self-Supervised Concept Discovery via Preference LearningS^2COPE: Self-Supervised Concept Discovery via Preference Learning
Jun 12, 2026Shilong Xiang, Zirui Zhang, Chengzhi MaoVision-Language Reasoning Visual Cues
PAWS: Preference Learning with Advantage-Weighted Segments
Jun 10, 2026Aleksandar Taranovic, Onur Celik, Niklas Freymuth +6Preference Learning Bird
A Regret Minimization Framework on Preference Learning in Large Language Models
Jun 8, 2026Suhwan Kim, Taehyun Cho, Geon-Hyeong Kim +4Reinforcement Learning From Human Feedback Preference Learning
ThoughtFold: Folding Reasoning Chains via Introspective Preference Learning
Jun 2, 2026Ziyan Liu, Xueda Shen, Yuzhe Gu +7Large Reasoning Models Chain-Of-Thought
Repair Before Veto: Repair-Augmented Constraint Learning for Contextual Decisions
Jun 1, 2026Yifan WangDecisions Abstention
Skill or Skip? Learning Selective Skill Invocation in Agentic Tasks via Dual-Granularity Preference Learning
May 30, 2026Chishui Chen, Jiaye Lin, Te Sun +6Agent Skill Libraries Agentic Reinforcement Learning
Isolating LLM Lexical Bias: A Curation-Free Triangulated Metric for Preference-Stage Learning
May 29, 2026Xiaoyang Ming, Jose Hernandez, Thomas Stephan JuzekLarge Language Model Bias Large Language Model Alignment
Federated Variational Preference Alignment with Gumbel-Softmax Prior for Personalized User Preferences
May 29, 2026Jabin Koo, Hoyoung Kim, Minwoo Jang +1Preference Alignment Pairwise Preferences
Reward Learning from Best-of- $N$ Preference Data: Targets, Tradeoffs, and Design PrinciplesReward Learning from Best-of-N Preference Data: Targets, Tradeoffs, and Design Principles
May 28, 2026Rattana Pukdee, Maria-Florina Balcan, Pradeep RavikumarBest-Of-N Preference Learning
Calibrated Preference Learning: The Case of Label Ranking
May 28, 2026Santo M. A. R. Thies, Viktor Bengs, Timo Kaufmann +2Generalizable Ranking Purpose Preference Learning
Hallucination Detection-Guided Preference Optimization for Clinical Summarization
May 27, 2026Shamanth Kuthpadi Seethakantha, Dung Ngoc Thai, Vara Prasad Gudi +6Summarization Hallucination Detection
Optimising Factual Consistency in Summarisation via Preference Learning from Multiple Imperfect Metrics
May 26, 2026Yuxuan Ye, Raul Santos-Rodriguez, Edwin SimpsonSummarization Facts
Active Query Synthesis for Preference Learning
May 25, 2026Namrata Nadagouda, Nauman Ahad, Maegan Tucker +1Preference Learning Active Learning
When In-Distribution Gains Fail: Evaluating Weak-to-Strong Reward Models under Preference Shift
May 25, 2026Khoi Le, Tri Cao, Phong Nguyen +5Preference Learning Preference Pairs
Learning Kernel-Based MDPs from Episodic Preferential Feedback
May 22, 2026Nikola Pavlovic, Sattar Vakili, Qing ZhaoReinforcement Learning From Human Feedback Preference Learning
Convex Optimization for Alignment and Preference Learning on a Single GPU
May 22, 2026Miria Feng, Mert PilanciPreference Learning Convex Optimization
Progressive Autonomy as Preference Learning: A Formalization of Trust Calibration for Agentic Tool Use
May 18, 2026Changkun OuTool-Schema Quality Predicts Tool Use Agent Decisions
Learning to Decide with AI Assistance under Human-Alignment
May 12, 2026Nina Corvelo Benz, Eleni Straitouri, Manuel Gomez-RodriguezArtificial Intelligence Alignment Preference Learning
SyncDPO: Enhancing Temporal Synchronization in Video-Audio Joint Generation via Preference Learning
May 12, 2026Xin Cheng, Xihua Wang, Ying Ba +5Joint Audio-Video Generation Speech-To-Text Alignment
Learning the Preferences of a Learning Agent
May 9, 2026Karim Abdel Sadek, Mark Bedaywi, Rhys Gould +1Preference Learning Optimal Policies
Compressed Video Aggregator: Content-driven Module for Efficient Micro-Video Recommendation
May 9, 2026Yang Xiao, Huiyuan Chen, Kaiyuan Deng +5Long Video Understanding Visual Embeddings
MoMo: Conditioned Contrastive Representation Learning for Preference-Modulated Planning
May 8, 2026Yusuf Syed, Viraj Parimi, Brian WilliamsClassical Planning Preference Learning
Multi-Objective Constraint Inference using Inverse reinforcement learning
May 7, 2026Syed Ihtesham Hussain Shah, Floris den Hengst, Aneta Lisowska +1Multi-Objective Reinforcement Learning Multi-Objective Constraint Inference
Preference Instability in Reward Models: Detection and Mitigation via Sparse Autoencoders
May 7, 2026Shunchang Liu, Xin Chen, Belen Martin Urcelay +1Preference Learning Sparse Autoencoders
PREFER: Personalized Review Summarization with Online Preference Learning
May 7, 2026Millend Roy, Agostino Capponi, Vineet GoyalRecommender Systems Summarization
PrefMoE: Robust Preference Modeling with Mixture-of-Experts Reward Learning
May 1, 2026Ziqin Yuan, Ruiqi Wang, Dezhong Zhao +2Preference Learning
Aligning with Your Own Voice: Self-Corrected Preference Learning for Hallucination Mitigation in LVLMs
Apr 27, 2026Byeonggeuk Lim, JungMin Yun, Junehyoung Kwon +2Large Language Model Hallucination Large Language Model Alignment
Probably Approximately Consensus: On the Learning Theory of Finding Common Ground
Apr 23, 2026Carter Blair, Ben Armstrong, Shiri Alouf-Heffetz +2Consensus Deliberation
Vibrotactile Preference Learning: Uncertainty-Aware Preference Learning for Personalized Vibration Feedback
Apr 22, 2026Rongtao Zhang, Xin Zhu, Masoume Pourebadi Khotbehsara +3Preference Learning Haptic Feedback
PREF-XAI: Preference-Based Personalized Rule Explanations of Black-Box Machine Learning Models
Apr 21, 2026Salvatore Greco, Jacek Karolczak, Roman Słowiński +1Explainable Artificial Intelligence Preference Learning
Cat-DPO: Category-Adaptive Safety Alignment
Apr 19, 2026Tiankai Yang, Yi Nian, Xinyuan Li +3Safety Alignment Large Language Model Alignment
Uncertainty-aware Generative Learning Path Recommendation with Cognition-Adaptive Diffusion
Apr 16, 2026Xiangrui Xiong, Hang Liang, Baiyang Chen +2Recommendation Architecture Uncertainty-Aware Teachers
Omni-RRM: Advancing Omni Reward Modeling via Automatic Rubric-Grounded Preference Synthesis
Jan 31, 2026Zicheng Kong, Dehua Ma, Zhenbo Xu +9Large Multimodal Models External Reward Models
N2M: Bridging Navigation and Manipulation by Learning Pose Preference from Rollout
Sep 23, 2025Kaixin Chai, Hyunjun Lee, Joseph J. LimRobotic Manipulation Policies Preference Learning
Interactive Multi-Objective Probabilistic Preference Learning with Soft and Hard Bounds
Jun 27, 2025Edward Chen, Sang T. Truong, Natalie Dullerud +2Pareto Frontier Preference Learning

Sep 11, 2026 · eess.ASJ/K move · Enter open · S save

Preference Optimization with LALM Feedback for Continuous Autoregressive Non-Verbal Vocalization Generation

Jingbin Hu, Qirui Zhan, Yuang Cao, Ziyu Zhang+6

We propose a preference optimization framework with Large Audio-Language Model (LALM) feedback for controllable non-verbal vocalization (NVV) generation in continuous autoregressive speech models. To construct preference data without human preference annotation, we build a bilingual prompt corpus by combining NVV-injected real transcripts with LLM-generated semantically aligned prompts, perform stochastic model rollouts, and use a LALM to rank candidate utterances and form same-prompt chosen--rejected pairs. We then adopt a two-stage optimization strategy: Rejection Sampling Fine-Tuning (RSFT) first adapts the model to LALM-selected high-scoring samples, followed by Anchored Flow-DPO, which formulates pairwise preference optimization using utterance-level flow-matching loss and retains the chosen-sample flow-matching objective as an SFT anchor. This design enables DPO-style preference learning without explicit sequence likelihoods while preserving direct supervision on preferred realizations. On the official 1,600-utterance NVVSpeech Challenge Track~2 test set, our method achieves a Final Track2Score of \textbf{75.80} (79.39 ZH / 72.21 EN), outperforming the VoxCPM2 baseline by \textbf{+1.84}. The improvements are mainly driven by higher NVV Accuracy and NVV Perceptual Effect, while Overall Quality remains stable.

Preference Learning Natural Language Feedback

Open paper Ask about it PDF