LLM Alignment

LLM: Large Language Model

Overview

LLM alignment studies how to shape language models’ behavior so they are helpful, truthful, safe, and responsive to the people and settings they serve. Common approaches include supervised fine-tuning, preference learning such as Direct Preference Optimization, reinforcement learning from human or AI feedback, and inference-time steering or retrieval. Researchers evaluate these methods on instruction-following, preference, jailbreak, factuality, multilingual, and multi-turn tasks, while also developing ways to represent disagreement among users and communities. A central challenge is that gains in safety or preference alignment may reduce usefulness, diversity, calibration, or capability, and desired behavior may not generalize to new prompts, groups, or deployment settings.

Latest papers

CardsList

634 papers24 papers in the last four weeks, down 64% on the four weeks before. 0.2% of all new papers.

  1. Foundations of Large Language Models

    Jan 16, 2025Tong Xiao, Jingbo ZhuLanguage Model PretrainingLLM Alignment

  2. Risk-Aware Alignment Must Incorporate Reward Uncertainty

    Oct 31, 2024Debangshu Banerjee, Kintan Saha, Aditya GopalanPolicy OptimizationLLM Alignment

  3. Decoupled Alignment for Robust Plug-and-Play Adaptation

    Jun 3, 2024Haozheng Luo, Jiahao Yu, Wenxin Zhang +9LLM AlignmentLLM Safety Alignment

  4. KTO: Model Alignment as Prospect Theoretic Optimization

    Feb 2, 2024Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff +2LLM AlignmentHuman Preference Modeling