When Optimization Meets Human Intent: Human-Centered Diffusion Model Optimization
Organizations: Peking University, China
Abstract
Diffusion models are evolving from general tools into personalized systems that adapt to user needs through fine-tuning and guidance. However, diffusion optimization mainly targets measurable objectives such as aesthetics, rarely examining whether they align with human intent. Human needs are multidimensional, contextual, and dynamic, whereas existing methods approximate them with fixed metrics. Because these metrics capture human needs only partially and may amplify deviations from true intent during optimization, higher scores need not yield results better matching human expectations. To understand how human needs enter and influence optimization, we systematically analyze how human intent is expressed, translated into optimizable objectives, applied to model behavior or generation, and evaluated through outputs. We further discuss trade-offs affecting human-intent alignment in personalized image generation and gaps between automated metrics and real user judgments. This human-centered synthesis reorganizes diffusion optimization and provides a systematic reference for personalized generation systems designed for human preferences.
Figures & tables
| Section | Proposed Table Title | Main Classification Dimensions |
|---|---|---|
| Sec. 4 | Stakeholders and Their Roles in Shaping Optimization Goals | End users, researchers, domain experts, and computational proxies. |
| Sec. 5 | Taxonomy of Human Intent Operationalization | Transformation of human intent into constraints, reward signals, preference models, and multi-objective optimization signals. |
| Sec. 6 | Taxonomy of Human-Centered Diffusion Optimization Methods | Reinforcement learning fine-tuning, direct preference optimization (DPO), training-free guidance, and latent/trajectory optimization. |
| Sec. 7 | Taxonomy of Optimization Trade-offs and Failure Modes | Reward hacking, diversity loss, human agency, and computational and interaction costs. |
| Sec. 8 | Taxonomy of Evaluation Practices for Human-Intent Alignment | Automated metrics, preference-based evaluation, human judgments, expert evaluation, and interactive and longitudinal evaluation. |
| Sec. 9 | Research Gaps and Future Directions | Adaptive intent modeling, dynamic objective coordination, user control, and interactive evaluation. |
| Survey | Primary Perspective | Organizing Framework | Central Research Focus |
|---|---|---|---|
| Zhang et al. ( Zhang et al., 2023 ) | Text-to-image diffusion | Generative foundations, methods, applications, and challenges | How diffusion models generate images from text |
| Jiang et al. ( Jiang et al., 2024b ) | Controllable diffusion generation | Control formulations, mechanisms, and evaluation | How diffusion generation can be guided by control signals |
| Cao et al. ( Cao et al., 2025a ) | Controllable text-to-image generation | Specific, multiple, and universal control conditions | How different conditions are incorporated into generation |
| Liu et al. ( Liu et al., 2026a ) | Human alignment of diffusion models | RLHF, DPO, test-time alignment, benchmarks, and evaluation | How diffusion models are aligned with human preferences |
| Wu et al. ( Wu et al., 2026b ) | Preference alignment in image generation and editing | Reward modeling, RLHF, DPO, applications, and challenges | How preference-alignment techniques improve generation and editing |
| This Review | Human-intent transformation | Intent representation Proxy objectives Optimization Trade-offs Human-centered validation | How human intent is represented, transformed, potentially distorted, and validated throughout optimization |
| ACM Digital Library & IEEE Xplore | |
|---|---|
| Title | ("diffusion model" OR "text-to-image diffusion" OR "diffusion-based image generation") |
| Text | ("reinforcement learning" OR "RLHF" OR "DPO" OR "preference optimization" OR "reward optimization" OR "training-free guidance" OR "inference-time optimization" OR "test-time optimization" OR "latent optimization" OR "trajectory optimization") AND ("human preference" OR "human feedback" OR "human intent" OR "user preference" OR "alignment" OR "personalization" OR "human-centered" OR "user interaction" OR "interactive generation" OR "iterative refinement" OR "human-AI co-creation") |
| Web of Science & Scopus | |
| Title | ("diffusion model" OR "text-to-image diffusion" OR "diffusion-based image generation") |
| Topic / Keywords | ("reinforcement learning" OR "RLHF" OR "DPO" OR "preference optimization" OR "reward optimization" OR "training-free guidance" OR "inference-time optimization" OR "latent optimization" OR "trajectory optimization") AND ("human preference" OR "human feedback" OR "human intent" OR "user preference" OR "alignment" OR "personalization" OR "human-centered" OR "interactive generation" OR "user interaction") |
| arXiv | |
| Intent category | Core concern | Typical optimization goals | Representative user needs / signals |
|---|---|---|---|
| Quality-Oriented | Improving the perceived quality of generated results | Aesthetic appeal, visual realism, perceptual quality, image attractiveness | “Make it look better,” “more realistic,” “more beautiful,” or preferences reflected in ratings or pairwise judgments |
| Fidelity-Oriented | Ensuring that outputs faithfully satisfy the user’s request or task specification | Text–image alignment, accurate depiction of objects, attributes, relations, and structural constraints | “Follow my prompt more accurately,” “preserve the object layout,” “edit only this part,” or instruction-following feedback |
| Preference-Oriented | Adapting generation to differences across users rather than optimizing only group-level averages | Personalized style preference, user-specific aesthetics, historical preference adaptation, identity-consistent generation | Personal choices, historical interactions, repeated feedback, user profiles, or multi-round preference refinement |
| Control-Oriented | Supporting how users interact with and steer the generation process, not only the final result | Structure preservation, local controllability, output diversity, exploration support, editability, interactive refinement | Region selection, reference images, iterative edits, requests for multiple alternatives, and needs for controllable exploration |
| Category | Sub-topic | Description | References |
| Intent Representation | Explicit Requirements and Constraints | Formalizes expressed needs as semantic attributes, structural constraints, or task conditions. | ( Xing et al., 2024 ) |
| Multimodal and Personalized Representations | Uses multimodal information, interaction histories, or personalized representations to encode user needs. | ( Lu et al., 2025 ; Dang et al., 2025 ; Li et al., 2026a ) | |
| Proxy Objective Construction | Reward and Preference Modeling | Converts human judgments into reusable reward functions or preference-based objectives. | ( Lee et al., 2025 ; Yuan et al., 2024 ; Ye et al., 2025 ; Wang et al., 2025a ; Huang et al., 2025a ; Luo et al., 2025 ; Liang et al., 2024 ; Xing et al., 2025b ; Croitoru et al., 2025 ; Li et al., 2025b ; Wang et al., 2025c ) |
| Predefined Computational Evaluators | Employs predefined evaluation criteria or learned computational representations as optimization proxies. | ( Yang et al., 2026 ) | |
| Objective Coordination | Multi-Objective Aggregation | Coordinates potentially competing objectives involving visual quality, fidelity, diversity, and user preferences. | ( Xie and Gong, 2025 ; Yan et al., 2026c ; Yan et al., ) |
| Proxy Mismatch | Human–Proxy Gap | Characterizes potential information loss or misalignment between human needs and their computational representations. | ( Karthik et al., 2025 ; Liu et al., 2025a ; Yang et al., 2024b ) |
| Category | Sub-topic | Description | Examples |
|---|---|---|---|
| Model-Level Optimization | Reinforcement Learning Fine-Tuning | Uses reward signals and policy updates to modify model parameters toward preferred generation behavior. | ( Yan et al., 2025 ; Hu et al., 2025 ; Jia et al., 2025 ; Miao et al., 2024 ) |
| Preference-Based Optimization | Learns from preferred and rejected outputs, often without an explicitly trained scalar reward model. | ( Liang et al., 2025 ; Wu et al., 2025 ; Na et al., 2025 ; Liu et al., 2025b ; Huang et al., 2026 ; Sun et al., 2025 ; Hong et al., 2026a ; An et al., 2025 ; Wang et al., 2026 ) | |
| Generation-Level Optimization | Training-Free Guidance | Modifies denoising or sampling directions using external objectives without updating base-model parameters. | ( Wallace et al., 2023 ; Park et al., 2025 ; Jeong et al., 2025 ; Kim et al., 2025a ) |
| Latent and Trajectory Optimization | Updates latent representations, explores candidate trajectories, or selects generation paths during inference. | ( Wei et al., 2026 ; Wu et al., 2026a ; Kim et al., 2025b ; Um and Ye, 2025 ; Bahng et al., 2025 ; Zhou et al., 2025 ; Simon et al., 2025 ; Jiang et al., 2025 ; Guo and Lin, 2024 ) |
| Category | Failure Mode / Trade-off | Human-Centered Implication | Supporting References |
|---|---|---|---|
| Proxy Fidelity | Reward Hacking and Overoptimization | Higher computational scores may not correspond to improved user satisfaction. | ( Zhang et al., 2025 ) |
| Generative Openness | Alignment–Diversity Trade-off | Stronger optimization may reduce output diversity and creative exploration opportunities. | ( Yun et al., 2025 ; Xing et al., 2025a ) |
| Human Agency | Automated Objective Control | Users may have limited ability to inspect, revise, or override objectives determined by the system. | Sec. 6.3 synthesis |
| System Cost | Optimization Gain vs. Computation | Additional training or inference computation may reduce interaction efficiency. | ( Mo et al., 2024 ; Guo et al., 2025a ) |
Diffusion Models
Diffusion models learn to reverse gradual data corruption, enabling flexible generation, editing, forecasting, and inference.
See all 1,266 papers