Multi-Objective Language Model Alignment

Latest papers 62

All topics
CardsList
  1. Multi-Objective Aligned Small Language Model Framework for SUD Patient Dialogue Generation

    Oct 6, 2026Thushara Manjari Naduvilakandy, Hyeju Jang, Mohammad Al HasanSmall Language ModelsUser Simulation

  2. Collaborative Personalized Preference Alignment for LLMs under Data Deficiency

    Oct 5, 2026Liyan Yang, Yige Yuan, Zhiqin YangPersonalized Language ModelsPersonalized Language Model Alignment

  3. Tag-Aware Structured Text Translation: Towards a Systematic Understanding

    Sep 24, 2026Zhanglin Wu, Hengchao Shang, Daimeng Wei +5Multi-Objective Language Model AlignmentMachine Translation

  4. Which Objectives Need a Dial? Predicting Objective Conflict and Covering Trade-offs in Steerable Pluralistic Alignment

    Sep 22, 2026David Tsoi, Esra DönmezDirect Preference OptimizationMulti-Objective Language Model Alignment

  5. Forty Shades of Blue: Quality-Diversity Alignment via Mode-Conditioned Reinforcement Learning

    Sep 14, 2026Jiayi Yuan, Hangoo Kang, James Jihao Liu +4RL for Language ModelsMulti-Objective Language Model Alignment

  6. Suan: Rectifying Direct Preference Safety Alignment in Large Language Models

    Sep 8, 2026Oleksandr Cherednichenko, Roman KlypaLLM AlignmentLLM Safety Alignment

  7. A Better Spur Should Start From Each Objective

    Sep 8, 2026Shanwen Mao, Hao Zhang, Guangtao nie +4Multi-Objective Reinforcement LearningGradient Interference

  8. Evolutionary Soups: Evolving Mixture-of-Experts for Multi-Objective LLM Alignment

    Aug 30, 2026Lingxiao Kong, Steffen Staab, Cong Yang +2Mixture-of-Experts Language ModelsMulti-Objective Language Model Alignment

  9. Automated Researchers Can Mitigate Well-characterized Alignment Failures

    Aug 28, 2026Chen Yueh-Han, Jiaxin Wen, Jan Hendrik KirchnerAI AlignmentLLM Safety Alignment

  10. MoPLEx: Estimating Plackett-Luce Mixture Models for Multi-Objective Alignment

    Aug 25, 2026Dongyue Li, Ziniu Zhang, Lu Wang +1Learning to RankDiscrete Choice Modeling

  11. Procedural Fairness Failures in RLHF from Preference Averaging

    Aug 10, 2026M P V S Gopinadh, Karthik Kamuju, Kummari Avinash +2Reward ModelingAlgorithmic Fairness

  12. Towards More Expressive Spoken LLMs: Fine-Grained Intent Benchmarking and Acoustic-Lexical Decoupled Policy Optimization

    Aug 4, 2026Xiang Lin, Tian-Hao Zhang, Chunfeng Wang +3RL for Language ModelsSpoken Dialogue Systems

  13. Beyond a Global Norm: Personalizing Toxicity Sensitivity in Language Models Without Retraining

    Jul 25, 2026Rares A. C. Diaconescu, Iulia Slanina, Alina Florea +5LLM AlignmentLLM Safety Alignment

  14. Improving LLM-Generated Process Model Quality Through Reinforcement Learning: The Role of Reward Function Design

    Jul 7, 2026Alexander Rombach, Chantale Lauer, Nijat MehdiyevRL for Language ModelsMulti-Objective Language Model Alignment

  15. Safe Inference-Time Alignment via Lagrangian Reward Augmentation

    Jul 2, 2026Yaswanth Chittepu, Ativ Joshi, Sohini Chintala +1LLM AlignmentLLM Safety Alignment

  16. Multi-Objective Exploration and Preference Optimization via Mutual Information

    Jul 1, 2026Hongyan Xie, Yikun Ban, Ruiyu Fang +4Multi-Objective Language Model AlignmentMutual Information Maximization

  17. AI Alignment From Social Choice Perspectives

    Jun 19, 2026Daniel Halpern, Evi Micha, Ariel D. Procaccia +3LLM AlignmentAI Alignment

  18. Beyond Uniform Forgetting: A Study of Sequential Direct Preference Optimization Across Preference Settings

    Jun 18, 2026Pranav Bhandari, Nicolas Fay, Amitava Datta +2Continual Learning for LLMsDirect Preference Optimization

  19. Steerable Cultural Preference Optimization of Reward Models

    Jun 17, 2026Minsik Oh, Advit Deepak, Sophie Wu +2Reward ModelingCultural Bias in Language Models

  20. Towards Pareto-Optimal Tool-Integrated Agents with Pareto Ranking Policy Optimization

    Jun 15, 2026Junyi Li, Xiaowei Qian, Yingyi Zhang +6Multi-Objective Reinforcement LearningMulti-Objective Language Model Alignment

  21. Understanding helpfulness and harmless tension in reward models

    Jun 11, 2026Eshaan Tanwar, Pepa AtanasovaReward ModelingLLM Alignment

  22. Multi-Faceted Interactivity Alignment in Full-Duplex Speech Models

    Jun 9, 2026Atsumoto Ohashi, Neil Zeghidour, Alexandre Défossez +1Spoken Dialogue SystemsMulti-Objective Language Model Alignment

  23. Pareto-Guided Teacher Alignment for Fair Personalized Text Generation

    Jun 8, 2026Tunazzina IslamLanguage Model Bias EvaluationMulti-Objective Language Model Alignment

  24. SAW: Stage-Aware Dynamic Weighting for Multi-Objective Reinforcement Learning in Large Language Models

    Jun 5, 2026Yuchen He, Baolong Bi, Shenghua Liu +7Multi-Objective Reinforcement LearningRL for Language Models

  25. DOG-DPO:Dynamic Optimization in Geometry for Safety Alignment

    Jun 4, 2026Yi Nian, Tiankai Yang, Yudi Zhang +7Direct Preference OptimizationLLM Safety Alignment

  26. What Do People Actually Want From AI? Mapping Preference Plurality

    Jun 4, 2026Julia Sepúlveda Coelho, Scott A. HaleLLM AlignmentHuman Preference Modeling

  27. TriAlign: Towards Universal Truth Consistency in Personalized LLM Alignment

    Jun 1, 2026Thi-Nhung Nguyen, Linhao Luo, Rollin Omari +3LLM AlignmentAlgorithmic Fairness

  28. Configurable Reward Model for Balanced Safety Alignment

    May 28, 2026Zhengping Jiang, Mehran Khodabandeh, Akash Bharadwaj +5Reward ModelingLanguage Model Safety Evaluation

  29. EvoRubric: Self-Evolving Rubric-Driven RL for Open-Ended Generation

    May 28, 2026Xin Guan, Xiaomeng Hu, Shen Huang +6Open-Ended GenerationRL for Language Models

  30. Multi-Stakeholder LLM Alignment: Decomposing Estimation from Aggregation

    May 26, 2026Lulu Zheng, Wenjin Yang, Xiangwen Zhang +4LLM AlignmentLLM-as-a-Judge

  31. MATO: Multi-objective Personalized Alignment with Test-time Optimization for Large Language Models

    May 25, 2026Linhao Luo, Thuy-Trang Vu, Van-Anh Nguyen +3LLM AlignmentInference-Time Optimization

  32. Spectral Souping: A Unified Framework for Online Preference Alignment

    May 19, 2026Yinlam Chow, Guy Tennenholtz, Ted Yun +4Multi-Objective Language Model AlignmentPreference Alignment

  33. General Preference Reinforcement Learning

    May 18, 2026Muhammad Umer, Muhammad Ahmed Mohsin, Ahsan Bilal +5Multi-Objective Language Model AlignmentRL Fine-Tuning

  34. Improving Code Translation with Syntax-Guided and Semantic-aware Preference Optimization

    May 13, 2026Yuhan Wu, Huan Zhang, Wei Cheng +3Code TranslationDirect Preference Optimization

  35. Pareto-Guided Optimal Transport for Multi-Reward Alignment

    May 13, 2026Ying Ba, Tianyu Zhang, Mohan Zhou +5Diffusion Model AlignmentReward Hacking

  36. BSO: Safety Alignment Is Density Ratio Matching

    May 12, 2026Tien-Phat Nguyen, Truong Nguyen, Thin Nguyen +3Functional Bregman DivergencesDirect Preference Optimization

  37. Explaining and Breaking the Safety-Helpfulness Ceiling via Preference Dimensional Expansion

    May 12, 2026ShiYing Huang, Liang Lin, Yuer Li +6LLM Safety AlignmentMulti-Objective Language Model Alignment

  38. Macro: Enhancing Multilingual Counterfactual Explanations through Alignment-as-Preference Optimization

    May 12, 2026Yilong Wang, Qianli Wang, Bohao Chu +3Multilingual Language ModelsCounterfactual Explanations

  39. ProteinOPD: Towards Effective and Efficient Preference Alignment for Protein Design

    May 11, 2026Yulin Zhang, He Cao, Zihao Jiang +6Protein Language ModelsMulti-Teacher Knowledge Distillation

  40. SalesSim: Benchmarking and Aligning Multimodal Language Models as Retail User Simulators

    May 8, 2026Yada Pruksachatkun, Elaine Wan, Lyanna Chen +2RL for Language ModelsLarge Language Model-Based Role-Play Simulation

  41. TUR-DPO: Topology- and Uncertainty-Aware Direct Preference Optimization

    Apr 30, 2026Abdulhady Abas Abdullah, Fatemeh Daneshfar, Seyedali Mirjalili +1LLM AlignmentDirect Preference Optimization

  42. A Multi-Dimensional Audit of Politically Aligned Large Language Models

    Apr 27, 2026Lisa Korver, Mohamed Mostagir, Sherief RedaLLM AuditingPolitical Bias in Language Models

  43. Pref-CTRL: Preference Driven LLM Alignment using Representation Editing

    Apr 26, 2026Imranul Ashrafi, Inigo Jauregi Unanue, Massimo PiccardiLLM AlignmentRepresentation Engineering

  44. C-MORAL: Controllable Multi-Objective Molecular Optimization with Reinforcement Alignment for LLMs

    Apr 24, 2026Rui Gao, Youngseung Jeon, Swastik Roy +2RL for Language ModelsMolecular Optimization

  45. MGDA-Decoupled: Geometry-Aware Multi-Objective Optimisation for DPO-based LLM Alignment

    Apr 22, 2026Andor Vári-Kakas, Ji Won Park, Natasa TagasovskaDirect Preference OptimizationGradient Descent

  46. Beyond Overlap Metrics: Rewarding Reasoning and Preferences for Faithful Multi-Role Dialogue Summarization

    Apr 19, 2026Xiaoyong Mei, Tingting Zuo, Da Chen +5Text SummarizationMulti-Objective Language Model Alignment

  47. WavAlign: Enhancing Intelligence and Expressiveness in Spoken Dialogue Models via Adaptive Hybrid Post-Training

    Apr 16, 2026Yifu Chen, Shengpeng Ji, Qian Chen +9Spoken Dialogue SystemsMulti-Objective Language Model Alignment

  48. Mitigating the Safety-utility Trade-off in LLM Alignment via Adaptive Safe Context Learning

    Feb 14, 2026Yanbo Wang, Minzheng Wang, Jian Liang +3LLM AlignmentRL for Language Model Reasoning

  49. VISPA: Pluralistic Alignment via Automatic Value Selection and Activation

    Jan 19, 2026Shenyan Zheng, Jiayou Zhong, Anudeex Shetty +3LLM AlignmentMulti-Objective Language Model Alignment