Preference Optimization

Ask about this topic

Momentum

13 papers in the last four weeks, up 30% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 94

All topics
CardsList

  1. Successive Training Stages and Large Language Model Persuasion: Effects of Misalignment, Supervised Fine-Tuning, and Preference Optimization

    Oct 7, 2026Antony Dalmiere, Pascal Marchand, Guillaume Auriol +1Persuasion in Language ModelsLLM Post-Training

    LAAS-TRUST · LAAS-TRUST, INSA Toulouse · LAAS-TSF, LAAS

  2. RobotAPO: Adversarial Physics Preference Optimization for Robotic Manipulation Video Generation

    Oct 7, 2026Kerui Li, Zhe Jing, Chenyi Huang +4Physical Consistency in Video GenerationRobotic Manipulation

    Institute of Automation, Chinese Academy of Sciences · GigaAI · Beijing Institute of Technology

  3. AuraSE: Low-Hallucination Generative Speech Enhancement via Multimodal Flow Matching and Inference Policy Optimization

    Oct 5, 2026Yingda Shen, Yao Qian, Yuxuan Hu +7Inference-Time OptimizationAudio Diffusion Models

    The Chinese University of Hong Kong, Shenzhen · Microsoft Research

  4. Rubric-Based Optimization for Text-to-Music Generation

    Oct 2, 2026Ping Wang, Guang Yang, Shao-Rong Su +3Text-to-Music GenerationRubric-Based RL

    Paul G. Allen School of Computer Science & Engineering, University of Washington · Department of Electrical and Computer Engineering, University of Washington · Allen Institute for AI

  5. Mitigating Social Sycophancy via Pluralistic Preference Optimization

    Oct 1, 2026Stephane Hatgis-Kessell, Myra Cheng, Xiaoxuan Hou +4LLM SycophancyPreference Optimization

    Department of Computer Science, Stanford University · Amazon · Department of Computer Science, University of Washington

  6. GAW-PO: Preference Optimization with Gradient-Aligned Token Weights

    Oct 1, 2026Andreea Dutulescu, Stefan Ruseti, Mihai Masala +2LLM AlignmentDirect Preference Optimization

    National University of Science and Technology POLITEHNICA Bucharest · NVIDIA

  7. Uncertainty-Normalized Margins for Direct Preference Optimization

    Sep 29, 2026Sadegh Khorasani, Petrus Mikkola, Matthias GrossglauserPairwise Preference LearningDirect Preference Optimization

    School of Computer and Communication Sciences, EPFL, Lausanne, Switzerland · University of Helsinki, Helsinki, Finland

  8. Preference-Guided Adaptation for Open-Vocabulary Semantic Segmentation via Prompt Disagreement

    Sep 28, 2026Hyun-Kurl Jang, Jihun Kim, Kuk-Jin YoonWeakly Supervised Semantic SegmentationPreference Optimization

    Visual Intelligence Lab. KAIST

  9. Agentic Multi-Turn Reasoning: A Fairness Approach

    Sep 27, 2026Thanh-Dat Truong, Sankalp Pandey, Hugh Churchill +3Preference OptimizationAgentic Reasoning

    CVIU Lab, University of Arkansas, USA · Dep. of Physics, University of Arkansas, USA · Dep. of Geosciences, University of Arkansas, USA +1

  10. A Zeroth-Order Paradigm for LLM Preference Alignment

    Sep 16, 2026Peter Chen, Xi Chen, Wotao Yin +1Pairwise Preference LearningLLM Alignment

    Department of Electrical Engineering and Computer Sciences (EECS) University of California, Berkeley Berkeley, CA 94720, USA · Stern School of Business New York University New York, NY 10012, USA · Decision Intelligence Lab (Seattle) DAMO Academy, Alibaba Group U.S. Bellevue, WA 98004, USA +1

  11. StalePO: Anchored Token-Level Preference Optimization using Legacy Post-Edits in Machine Translation

    Sep 14, 2026Rohit Dhaipule, Sukhdeep Singh Kharbanda, Prasanth Bathala +2Preference OptimizationLanguage Model Post-Training

    Translation Services, Amazon

  12. Direct Preference Density Alignment for Conversational Audio Equalization

    Sep 14, 2026Ioannis Stylianou, Sven Ewan Shepstone, Jon Francombe +2RL for Language ModelsLLM Alignment

    Department of Electronic Systems Aalborg University Aalborg, Denmark · Bang & Olufsen A/S Struer, Denmark

  13. Direct Diversity Optimization for Diverse Successful Trajectories in Preference Post-Training

    Sep 9, 2026Junwon Ko, Dong-Jae Lee, Minchan Kwon +2Offline RLPreference Optimization

    School of Electrical Engineering, KAIST Daejeon, Republic of Korea

  14. Suan: Rectifying Direct Preference Safety Alignment in Large Language Models

    Sep 8, 2026Oleksandr Cherednichenko, Roman KlypaLLM AlignmentLLM Safety Alignment

    Department of Mathematics and Mathematical Statistics Integrated Science Lab, Umeå University Umeå, Sweden · Univ. Grenoble Alpes, CNRS, Grenoble INP, LJK 38000 Grenoble, France

  15. TV-Regulated OPD: Direction Matters in On-Policy Distillation

    Sep 8, 2026Han Xiao, Yifan Niu, Dongyi Liu +2Preference OptimizationOn-Policy Distillation

    The Hong Kong University of Science and Technology (Guangzhou) · The University of Edinburgh · The Hong Kong University of Science and Technology

  16. CUNO: Curriculum and Preference Optimization for Stable Graph Unlearning under Mass Deletion

    Sep 8, 2026Chenhan Zhang, Ali Braytee, Madhushi Bandara +4Preference OptimizationMachine Unlearning

    University of Technology Sydney

  17. MARBO: Relational Belief Grounding for LLM Agents in Social Deduction Games

    Sep 6, 2026Yechan Hwang, Sangjun Bae, Jeongmo Kim +2Game-Playing AgentsSocial Deduction Games

    Graduate School of Artificial Intelligence, UNIST, Ulsan, South Korea.

  18. MCPO: Modality-Contrastive Preference Optimization for Multimodal Chain-of-Thought Compression

    Sep 4, 2026Guangheng Yang, Zhenliang Ni, Zhenkai Wu +4Efficient Multimodal InferenceMultimodal Large Language Models

    Huawei Technologies Ltd. Shanghai, China

  19. FiMI Banking: A Sovereign Model for Indian Retail Banking

    Sep 3, 2026NPCI AI Research Team, Aman Kumar, Asit Desai +15Financial ServicesRL for Language Models

    NPCI AI Research Team

  20. CoMerge: Conflict-Driven Preference Optimization for Multi-Task Model Merging

    Sep 2, 2026Mingjie Zheng, Zihao Chen, Wenqing Chen +4Model MergingPreference Optimization

    School of Software Engineering, Sun Yat-sen University, Zhuhai, China · The State Key Laboratory of Blockchain and Data Security, Zhejiang University, Hangzhou, China · School of Artificial Intelligence, Sun Yat-sen University, Zhuhai, China +1

  21. Ready to Speak: Aligning LLMs for TTS-Friendly Text Generation

    Sep 1, 2026Thibaut Thonet, Jos Rozen, Laurent BesacierLLM AlignmentTTS Evaluation

    NAVER LABS Europe

  22. Sycophantic Agreement Transfers with Neutral Data via Contrastive Preference Optimization

    Aug 31, 2026Camila Blank, Zhuofan Ying, Christopher Potts +2LLM SycophancyEmergent Misalignment in Language Models

    1Stanford University · 2Columbia University

  23. Balancing Privacy, Utility, and Safety in LLM Alignment through Preference Optimization

    Aug 31, 2026Dishu Yang, Jingjing Liu, Jize LiLLM AlignmentMemorization in Language Models

    Khoury College of Computer Sciences Northeastern University San Jose, USA · Department of Electrical Engineering and Computer Sciences University of California, Berkeley Berkeley, USA · Department of Economics Boston University Boston, USA

  24. VoxAudio: Vocalized Audio Synthesis via Multi-Reward Autoregressive Flow Matching

    Aug 13, 2026Wenxiang Guo, Changhao Pan, Ziyue Jiang +2Text-to-Audio GenerationPreference Optimization

    Zhejiang University, Hangzhou, China

  25. SSPO: Structure-Aware Similarity-Weighted Preference Optimization for Neural Combinatorial Optimization

    Aug 12, 2026Yuanyu Li, Jintao Xu, Zijiang Liu +6Combinatorial OptimizationPreference Optimization

    Supply Chain Tech Team Y, JD.com, Beijing, China · Department of Industrial Engineering, Tsinghua University, Beijing, China · Faculty of Engineering and Faculty of Business and Economics, The University of Hong Kong, Hong Kong, China

  26. TSPORec: Token Selection via Preference Optimization for LLM-Based Sequential Recommendation

    Aug 10, 2026Wenqiao Zhu, Chao Xu, Haipang Wu +1LLM Inference EfficiencySequential Recommendation

  27. Learning Sexism Detection Using Multi-Agent Perspectivist Preference Optimization

    Aug 4, 2026Hadi Mohammadi, Tina Shahedi, Robert A. Bagheri +2Annotator DisagreementPreference Optimization

    Department of Methodology and Statistics, Utrecht University, The Netherlands · Department of Information and Computing Sciences, Utrecht University, The Netherlands · Field Technology Innovations (FTI) & Robotics, Wageningen University & Research, The Netherlands

  28. Adaptive Human-Robot Collaborative Painting Combining Preference-Based Optimization and Dynamic Motion Primitives

    Aug 3, 2026C. Cella, M. Ristic, M. Faroni +2Robot Motion GenerationHuman-Robot Collaboration

    Politecnico di Milano

  29. Cloud-ScPO: Hidden-State Geometry for Semi-Supervised Preference Optimization in LLM Reasoning

    Aug 2, 2026Yuzhou Liu, Xiyang HuPreference OptimizationSemi-Supervised Learning

    University of Southern California · Arizona State University

  30. Rolling With Resistance: Preference-Optimized LLM Counselors Can Trade Goal Persistence for Relational Attunement in Motivational Interviewing

    Jul 30, 2026Weiying Chen, Junlong Shen, Zhexuan TangDirect Preference OptimizationPreference Optimization

    University of Alberta · University of Shanghai for Science and Technology

  31. Temporal Concentration from Rollout Errors: Implicit Preference Optimization for Text-to-Video Diffusion

    Jul 30, 2026Henglin Liu, Fangyuan Kong, Jing Wang +7Diffusion Model AlignmentVideo Diffusion Models

    Tsinghua University · Kling Team, Kuaishou Technology · Sun Yat-sen University

  32. AutoPref: Automatic Discovery of Task-Specific Preference Objectives for Neural Combinatorial Optimization

    Jul 30, 2026Shengda Gu, Kai Li, Xinyi Ke +3Automated Algorithm DiscoveryPairwise Preference Evaluation

    C2DL, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · Tencent AI Lab +1

  33. BridgeAlign: Bridging Preference Alignment for Humanities and Social Sciences

    Jul 29, 2026Ru Peng, Haokai Xu, Xijun Gu +11Pairwise Preference LearningPreference Optimization

    Zhejiang University · Inclusion AI, Ant Group · Qwen Team, Alibaba Group +2

  34. TELLER: Dual-Path Iterative Preference Optimization for Table Entity Linking

    Jul 29, 2026Yixin Peng, Kehao Li, Stefan DeckerDirect Preference OptimizationPreference Optimization

    RWTH Aachen, Ahornstraße 55, 52074 Aachen, Germany

  35. Less Data, Better Alignment: Data-Centric Multi-Evaluator Agreement for Preference Optimization

    Jul 27, 2026Zhengtao Yao, Runhao Li, Xupeng Chen +12Preference Optimization

    University of Southern California · New York University · Columbia University +4

  36. EvoThink: Evolving Thinking in Large Reasoning Models via Self-Pruning and Aha-Moment Preference Optimization

    Jul 22, 2026Xinbang Dai, Zheyu Xin, Huikang Hu +7Overthinking in Language ModelsSelf-Training for Language Models

    Southeast University · Noah’s Ark Lab

  37. RIMS: Preference Optimization via Smoothed Multi-pair Aggregation for Small-Scale LLM Retrieval-Augmented Generation

    Jul 17, 2026Pei Tian, Zihan Dong, Tianci Liu +2Retrieval-Augmented GenerationSmall Language Models

    Department of Computer Science, Columbia University · Department of Statistics, Rutgers University · School of Electrical and Computer Engineering, Purdue University +1

  38. TD-DPO: Difference-Aware Preference Optimization for Mitigating Sycophancy in Clinical Autism Intervention Dialogue

    Jul 17, 2026Shuzhong Lai, Junhong Lai, Chenxi Li +5LLM SycophancyDirect Preference Optimization

    1Nanhu Brain-Computer Interface Institute · 2MOE Frontiers Science Center for Brain and Brain-Machine Integration, Zhejiang University · College of Computer Science and Technology, Zhejiang University +3

  39. Groc-PO: Grounded Context Preference Optimization for Truthful Multimodal LLMs

    Jul 15, 2026Zhixiao Zheng, Zheren Fu, Zhiyuan Yao +3Multimodal GroundingMultimodal Large Language Models

    University of Science and Technology of China Hefei, China · Xiaomi Corporation Beijing, China · State Key Laboratory of Communication Content Cognition, People’s Daily Online Beijing, China

  40. Exploring Post-Training Alignment of Small Language Models for Biomedical Data-to-Text Generation: A Case Study of Medication Leaflet

    Jul 15, 2026Xi Yang, Guodong Liu, Chuqin Li +10LLM AlignmentSmall Language Models

    Advanced Intelligence, Eli Lilly and Company, Indianapolis, IN, USA · Department of Biostatistics and Health Data Science, Indiana University, Indianapolis, IN, USA · Center for Biomedical Informatics, Regenstrief Institute, Indianapolis, IN, United States +2

  41. Direct Image-to-Modern Vietnamese Translation of Han-Nom Manuscripts via Multimodal RLHF Preference Alignment

    Jul 13, 2026Thi Kim Trang Vo, Nghia Hieu Nguyen, Ha Minh TanDirect Preference OptimizationPreference Optimization

    University of Information Technology, Ho Chi Minh City, Vietnam · Vietnam National University, Ho Chi Minh City, Vietnam

  42. PORTS: Preference-Optimized Retrievers for Tool Selection with Large Language Models

    Jul 3, 2026Lorenzo Molfetta, Giacomo Frisoni, Nicolò Monaldini +1Preference OptimizationLLM Tool Use

    Department of Computer Science and Engineering, University of Bologna

  43. KARMA: Knowledge graph-based Automated Reasoning Materialization and Alignment

    Jul 3, 2026Jinkyeong Choi, Chaebin Jeong, Donghyeon ParkContrastive LearningPreference Optimization

    Sejong University

  44. Distributionally Robust Listwise Preference Optimization

    Jul 2, 2026Xudong Wu, Jian Qian, Pangpang Liu +2LLM AlignmentDistributionally Robust Optimization

    The University of Hong Kong · Hong Kong SAR · Yale University +3

  45. RSICCLLM: A Multimodal Large Language Model for Remote Sensing Image Change Captioning

    Jun 26, 2026Yelin Wang, Zijia Song, Shuo Ye +6Remote Sensing Image UnderstandingVLM Adaptation

    State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · Harbin Institute of Technology, Shenzhen, China · Great Bay University, Dongguan, China +1

  46. MORPH: Generative Retrieval via Diffusion Transformer with Metric-Ordered Sequence Training and Hybrid-Policy Preference Optimization

    Jun 25, 2026Chenghao Liu, Yu Zhang, Zhongtao Jiang +8Generative RetrievalPreference Optimization

    Peking University

  47. OracleAnalyser: Analysing Implicit Semantics of Oracle Bone Scripts through MLLMs with Post-training

    Jun 24, 2026Zijia Song, Yelin Wang, Zhengyi Ma +5VLM ReasoningPreference Optimization

    National University of Defense Technology · ShanghaiTech University · Great Bay University +1

  48. Against Proxy Optimization

    Jun 22, 2026Sven NethPreference OptimizationDecision-Making under Uncertainty

    University of Pittsburgh

  49. Beyond Uniform Forgetting: A Study of Sequential Direct Preference Optimization Across Preference Settings

    Jun 18, 2026Pranav Bhandari, Nicolas Fay, Amitava Datta +2Continual Learning for LLMsDirect Preference Optimization

    1Network Analysis and Social Influence Modelling (NASIM) Lab · School of Physics Maths and Computing, The University of Western Australia · School of Psychological Science, The University of Western Australia +1

  50. Temporal Preference Optimization for Unsupervised Retrieval

    Jun 16, 2026HyunJin Kim, Jaejun Shim, Young Jin Kim +1Temporal IRPreference Optimization

    1Sungkyunkwan University, Suwon, South Korea · 2Microsoft, Redmond, USA

  51. Effects of Objective Normalization on Regions of Interest in Preference-Based Evolutionary Multi-Objective Optimization

    Jun 15, 2026Ryuichi Mogami, Ryoji TanabeMulti-Objective Evolutionary OptimizationPreference Optimization

    Yokohama National University, Yokohama, Japan

  52. PVminerLLM2: Improving Structured Extraction of Patient Voice via Preference Optimization

    Jun 15, 2026Samah Fodeh, Linhai Ma, Ganesh Puthiaraju +7HealthcareClinical Information Extraction

    Department of Emergency Medicine, Yale School of Medicine, 464 Congress Ave, 06519, CT, USA · Department of Social and Behavioral Sciences, Yale School of Public Health, 60 College Street, 06520, CT, USA · Division of Research, Texas State University, 601 University Dr., 78666, TX, USA

  53. PolyAlign: Conditional Human-Distribution Alignment

    Jun 11, 2026L. D. M. S. Sai Teja, Ufaq Khan, Sathira Silva +2Multilingual Language ModelsLLM Alignment

    NIT Silchar, India · MBZUAI, Abu Dhabi, UAE

  54. Emo-LiPO: Listwise Preference Optimization for Fine-Grained Emotion Intensity Control in LLM-based Text-to-Speech

    Jun 11, 2026Yihang Lin, Li Zhou, Congwei Cao +4TTS SynthesisPreference Optimization

    The Chinese University of Hong Kong, Shenzhen · Shenzhen Loop Area Institute · Agency for Science, Technology and Research +2

  55. SAGE: Answer-Conditioned Uncertainty Targets for Verbal Uncertainty Alignment

    Jun 9, 2026Kaiwen Shi, Zheyuan Zhang, Yanfang YeConfidence Estimation in Language ModelsPreference Optimization

    University of Notre Dame

  56. FoA-SR: Faithful or Aesthetic? Profile-Aware Preference Optimization for Real-World Image Super-Resolution

    Jun 9, 2026Amjad Mahdi Alqarni, Peizhong JuDiffusion Models for Image RestorationPreference Optimization

    Department of Computer Science University of Kentucky Lexington, KY, USA

  57. ThoughtFold: Folding Reasoning Chains via Introspective Preference Learning

    Jun 2, 2026Ziyan Liu, Xueda Shen, Yuzhe Gu +7Efficient Language Model ReasoningPreference Optimization

    Shanghai Artificial Intelligence Laboratory · University of Science and Technology of China · MMLab, The Chinese University of Hong Kong

  58. Drifting Preference Optimization for One-Step Generative Models

    Jun 1, 2026Zhou Jiang, Yandong Wen, Zhen LiuDiffusion Model AlignmentOne-Step Generative Modeling

    Westlake University · The Chinese University of Hong Kong, Shenzhen

  59. From "Weak" Signals to Strong Models: Preference Delta Aggregation with LoRA Merging

    May 29, 2026Qi Sun, Siyue Zhang, Yulin Chen +3Pairwise Preference LearningLow-Rank Adaptation

    NYU Shanghai · NTU · NYU +1

  60. Human Label Variation as Stable Signal: Learning Annotator-Specific Explanation Behavior via Cross-Annotator Preference Optimization

    May 27, 2026Beiduo Chen, Pingjun Hong, Ziyun Zhang +3LLM-Assisted AnnotationPreference Optimization

    U MaiNLP, Center for Information and Language Processing, LMU Munich, Germany · Æ Munich Center for Machine Learning (MCML), Munich, Germany · [ University of Vienna, Austria +1