Sequential Preference Optimization

Follow this topic Ask about this topic

Momentum

4 papers in the last four weeks, up 33% on the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 32

All topics

Preference-Guided Adaptation for Open-Vocabulary Semantic Segmentation via Prompt Disagreement
Sep 28, 2026Hyun-Kurl Jang, Jihun Kim, Kuk-Jin YoonOpen-Vocabulary Segmentation Semantic Segmentation
Visual Intelligence Lab. KAIST
A Zeroth-Order Paradigm for LLM Preference Alignment
Sep 16, 2026Peter Chen, Xi Chen, Wotao Yin +1Preference Alignment Sequential Preference Optimization
Department of Electrical Engineering and Computer Sciences (EECS) University of California, Berkeley Berkeley, CA 94720, USA · Stern School of Business New York University New York, NY 10012, USA · Decision Intelligence Lab (Seattle) DAMO Academy, Alibaba Group U.S. Bellevue, WA 98004, USA +1
DiaWhisper-DPO: Role-Attributed Transcription of Clinical Interviews via Failure-Mined Preference Optimization
Sep 15, 2026Weiming Li, Ana Catarina Fidalgo Barata, Miguel Constante +1Speaker Diarization Transcript
Institute for Systems and Robotics (ISR), LARSyS, Departamento de Bioengenharia, Instituto Superior Técnico (IST), Universidade de Lisboa, 1049-001 Lisboa, Portugal · Hospital Beatriz Ângelo, Faculdade de Medicina Universidade Católica Portuguesa, 2674-514 Loures, Portugal
Suan: Rectifying Direct Preference Safety Alignment in Large Language Models
Sep 8, 2026Oleksandr Cherednichenko, Roman KlypaSafety Alignment Large Language Model Alignment
Department of Mathematics and Mathematical Statistics Integrated Science Lab, Umeå University Umeå, Sweden · Univ. Grenoble Alpes, CNRS, Grenoble INP, LJK 38000 Grenoble, France
Low-Resource Preference Adaptation of LLMs via Activation-Based Label Propagation
Aug 31, 2026Alessio Galatolo, Meriem BeloucifSequential Preference Optimization Preference Alignment Learning
Uppsala University
Context Blindness in DPO: Mitigating Object Hallucination in MLLMs via Context-Calibrated Preference Optimization
Aug 12, 2026Byungoh Ko, Jinyoung Park, Jongha Kim +3Hallucination Mitigation Object Hallucination
Korea University, Seoul, Republic of Korea · KAIST, Daejeon, Republic of Korea
TSPORec: Token Selection via Preference Optimization for LLM-Based Sequential Recommendation
Aug 10, 2026Wenqiao Zhu, Chao Xu, Haipang Wu +1Multimodal Recommendation Model Recommendation Quality
Linguistically-Aligned and Visually-Grounded Preference Optimization for Clinically-Augmented Medical Report Generation
Aug 9, 2026Qiang Hu, Yuxuan Luo, Yingjie Guo +4Medical Report Generation Multimodal Clinical Data
Huazhong University of Science and Technology
Positive-Unlabeled Preference Optimization For Chest X-ray Report Generation
Aug 5, 2026Yuta Kobayashi, Pradyun Ramesh, Muhammad Ahmed Chaudhry +5Radiology Report Generation Medical Vision-Language Models
Columbia University · Stanford University · Emory University
Cloud-ScPO: Hidden-State Geometry for Semi-Supervised Preference Optimization in LLM Reasoning
Aug 2, 2026Yuzhou Liu, Xiyang HuSequential Preference Optimization LLM Reasoning Strategies
University of Southern California · Arizona State University
Temporal Concentration from Rollout Errors: Implicit Preference Optimization for Text-to-Video Diffusion
Jul 30, 2026Henglin Liu, Fangyuan Kong, Jing Wang +7Text-To-Video Diffusion Models Video Diffusion Models
Tsinghua University · Kling Team, Kuaishou Technology · Sun Yat-sen University
EvoThink: Evolving Thinking in Large Reasoning Models via Self-Pruning and Aha-Moment Preference Optimization
Jul 22, 2026Xinbang Dai, Zheyu Xin, Huikang Hu +7Large Reasoning Models Redundancy
Southeast University · Noah’s Ark Lab
TD-DPO: Difference-Aware Preference Optimization for Mitigating Sycophancy in Clinical Autism Intervention Dialogue
Jul 17, 2026Shuzhong Lai, Junhong Lai, Chenxi Li +5Autism Spectrum Disorder Sycophancy
1Nanhu Brain-Computer Interface Institute · 2MOE Frontiers Science Center for Brain and Brain-Machine Integration, Zhejiang University · College of Computer Science and Technology, Zhejiang University +3
Distributionally Robust Listwise Preference Optimization
Jul 2, 2026Xudong Wu, Jian Qian, Pangpang Liu +2Sequential Preference Optimization Backtranslation Augmented Direct Preference Optimization
The University of Hong Kong · Hong Kong SAR · Yale University +3
Clearer Sight, Fewer Lies: Oriented Pickup Preference Optimization for Multimodal Hallucination Mitigation
Jun 29, 2026Xin Zou, Haolin Deng, Yibo Yan +5Multimodal Large Language Models Sequential Preference Optimization
HKUST (GZ) · HKUST · OPPO AI Center
Steerable Cultural Preference Optimization of Reward Models
Jun 17, 2026Minsik Oh, Advit Deepak, Sophie Wu +2Large Language Model Alignment Sequential Preference Optimization
1Stanford University · University of Amsterdam
Towards Pareto-Optimal Tool-Integrated Agents with Pareto Ranking Policy Optimization
Jun 15, 2026Junyi Li, Xiaowei Qian, Yingyi Zhang +6Sequential Preference Optimization Large Language Model Alignment
Department of Data Science, City University of Hong Kong, Hong Kong, China · Zhejiang University of Technology, Hangzhou, China · Huawei Technologies Noah’s Ark Lab, Hong Kong, China
Analyzing and Improving Fine-grained Preference Optimization in Medical LVLMs
Jun 10, 2026Shayan Mohammadizadehsamakosh, Pritam Sarkar, Leonid Sigal +2Medical Vision-Language Models Vision-Language Alignment
York University · Vector Institute · University of British Columbia +1
A Regret Minimization Framework on Preference Learning in Large Language Models
Jun 8, 2026Suhwan Kim, Taehyun Cho, Geon-Hyeong Kim +4Reinforcement Learning From Human Feedback Sequential Preference Optimization
Seoul National University, Seoul, South Korea · LG AI Research, Seoul, South Korea · UNIST, Ulsan, South Korea +1
Local Preferential Bayesian Optimization
Jun 1, 2026Johanna Menn, Miriam Kober, Paul Brunzema +2Multi-Objective Bayesian Optimization Algorithms Bayesian Optimization
Institute for Data Science in Mechanical Engineering, RWTH Aachen University, Aachen, Germany · Department of Clinical Research, University of Bern, Bern, Switzerland · Center for Reproducible Science and Research Synthesis, University of Zurich, Zurich, Switzerland +1
Reasoning Matters: Mitigate Hallucination in Multimodal Large Reasoning Models via Reasoning-Conditioned Preference Optimization
May 27, 2026Jiawei Kong, Hao Fang, Shunxiang Liao +5Multimodal Reasoning Hallucination Mitigation
Tsinghua Shenzhen International Graduate School, Tsinghua University · Harbin Institute of Technology, Shenzhen
Semantic Reward Collapse and the Preservation of Epistemic Integrity in Adaptive AI Systems
May 12, 2026William ParrisEpistemic Integrity Rationality
BDB Labs / BagelTech · San Diego, California, USA
TokenRatio: Principled Token-Level Preference Optimization via Ratio Matching
May 12, 2026Truong Nguyen, Tien-Phat Nguyen, Linh Ngo Van +3Sequential Preference Optimization Token-Level Uncertainty
Hanoi University of Science and Technology · University of Stuttgart · German Research Center for Artificial Intelligence +3
Confidence-Aware Alignment Makes Reasoning LLMs More Reliable
May 8, 2026Kejia Chen, Jiawen Zhang, Yihong Wu +5LLM Reasoning Strategies Sequential Preference Optimization
Zhejiang University · Université de Montréal · Sun Yat-sen University +1
Topology-Enhanced Alignment for Large Language Models: Trajectory Topology Loss and Topological Preference Optimization
May 8, 2026Yurui Pan, Ke Xu, Bo PengLarge Language Model Alignment Sequential Preference Optimization
School of Computing and Intelligent Innovation, Fudan University · School of Economics and Management, Tongji University · College of Information Technology, Shanghai Ocean University
Arena as Offline Reward: Efficient Fine-Grained Preference Optimization for Diffusion Models
May 7, 2026Zhikai Li, Yue Zhao, Edward Zhongwei Zhang +4Diffusion-Based Reinforcement Learning Methods Diffusion Models
Institute of Automation, Chinese Academy of Sciences · University of California, Berkeley · University of California, Santa Barbara
A Finite Time Analysis of Thompson Sampling for Bayesian Optimization with Preferential Feedback
Apr 27, 2026Joseph Lazzaro, Davide Buffelli, Da-shan Shiu +1Thompson Sampling Bayesian Optimization
Imperial College London · MediaTek Research
Hindsight Preference Optimization for Financial Time Series Advisory
Apr 27, 2026Yanwei Cui, Guanghui Wang, Xing Zhang +7Sequential Preference Optimization Large Language Model Adaptation
1AWS Generative AI Innovation Center · 2HSBC Holdings Plc. HSBC Technology Center China
HiPO: Hierarchical Preference Optimization for Adaptive Reasoning in LLMs
Apr 22, 2026Darsh Kachroo, Arjun Prasaath Anbazhagan, Adriana Caraeni +2Sequential Preference Optimization LLM Reasoning Strategies
Vellore Institute of Technology, Chennai · Northwestern University · University of Massachusetts Amherst +2
S2H-DPO: Hardness-Aware Preference Optimization for Vision-Language Models
Apr 20, 2026Nitish Shukla, Surgan Jandial, Arun RossMultiple Vision Tasks Sequential Preference Optimization
Michigan State University · Carnegie Mellon University
Towards Disentangled Preference Optimization Dynamics: Suppress the Loser, Preserve the Winner
Apr 20, 2026Wei Chen, Yubing Wu, Junmei Yang +5Sequential Preference Optimization Incentives
1South China University of Technology · 2AIP, RIKEN · 3Columbia University +1
Shapley-based Data Valuation for LLM Alignment via Sequential Preference Optimization
Dec 12, 2025Mélissa Tamine, Otmane Sakhi, Benjamin Heymann +2Large Language Model Alignment Sequential Preference Optimization
Criteo AI lab, FairPlay joint team, France · CREST, ENSAE, Institut Polytechnique de Paris · Criteo AI Lab, France +1

Sep 28, 2026, cs.CVJ/K move · Enter open · S save

Preference-Guided Adaptation for Open-Vocabulary Semantic Segmentation via Prompt Disagreement

Hyun-Kurl Jang, Jihun Kim, Kuk-Jin Yoon

Visual Intelligence Lab. KAIST

Open-Vocabulary Segmentation Semantic Segmentation Open-Vocabulary Vision-Language Model Adaptation Parameter-Efficient Adaptation Sequential Preference Optimization+2

Open paper AskPDF arXiv

Open-vocabulary semantic segmentation (OVSS) enables pixel-level prediction over arbitrary text-specified vocabularies and has shown strong generalization on common benchmarks. However, OVSS performance often degrades in specialized domains such as medical imaging, remote sensing, and industrial inspection, where dense pixel-level masks for adaptation are costly to obtain and require domain-specific expertise. We propose a preference-guided adaptation framework that replaces dense mask supervision with binary preferences. We observe that different prompt templates produce systematically different segmentations for the same image, a phenomenon we call prompt disagreement, and we repurpose it as a built-in source of preference supervision. Building on this, we mine localized preference queries from regions of high cross-template uncertainty, and adapt the OVSS model with Region-Localized Preference Optimization (RLPO) together with consistency regularization that stabilizes updates outside the queried region. Across extensive experiments on the MESS benchmark, the proposed method achieves consistent gains across diverse OVSS backbones without any pixel-level annotation, and remains effective under noisy preferences. Our code is available at https://github.com/blue-531/pref-ovss.

Momentum

Related topics

Latest papers 32