Causal Direct Preference Optimization for Distributionally Robust Generative Recommendation

Chu Zhao; Enneng Yang; Jianzhe Zhao; Guibing Guo

cs.IRMar 21, 2026

Causal Direct Preference Optimization for Distributionally Robust Generative Recommendation

Authors: Chu Zhao, Enneng Yang, Jianzhe Zhao, Guibing Guo

Organizations: Northeastern University, Shenyang, China · Shenzhen Campus of Sun Yat-sen University, China

Abstract

Direct Preference Optimization (DPO) guides large language models (LLMs) to generate recommendations aligned with user historical behavior distributions by minimizing preference alignment loss. However, our systematic empirical research and theoretical analysis reveal that DPO tends to amplify spurious correlations caused by environmental confounders during the alignment process, significantly undermining the generalization capability of LLM-based generative recommendation methods in out of distribution (OOD) scenarios. To mitigate this issue, we propose CausalDPO, an extension of DPO that incorporates a causal invariance learning mechanism. This method introduces a backdoor adjustment strategy during the preference alignment phase to eliminate interference from environmental confounders, explicitly models the latent environmental distribution using a soft clustering approach, and enhances robust consistency across diverse environments through invariance constraints. Theoretical analysis demonstrates that CausalDPO can effectively capture users stable preference structures across multiple environments, thereby improving the OOD generalization performance of LLM-based recommendation models. We conduct extensive experiments under four representative distribution shift settings to validate the effectiveness of CausalDPO, achieving an average performance improvement of 17.17% across four evaluation metrics.

Figures & tables

Figure 1: Left : This figure presents the number of interactions (frequency) for DPO-based models across item popularity groups, with popularity decreasing from G1 (head) to G5 (tail). Middle : This figure illustrates how LLM-based recommendation models can learn and amplify spurious correlations during preference alignment, and how the DPO mechanism further reinforces these spurious correlations. Right : Using a Structural Causal Model (SCM), this figure analyzes how environmental confounders $E$ affect the model and demonstrates how their influence can be mitigated via the backdoor adjustment criterion.

Methods	Yelp2018				ML-10M				Book-Crossing
Methods	H@10	N@10	H@20	N@20	H@10	N@10	H@20	N@20	H@10	N@10	H@20	N@20
SASRec	1.01	0.43	1.51	0.61	0.50	0.02	0.84	0.30	0.33	0.21	0.68	0.21
BIGRec	0.77	0.50	1.28	0.62	0.53	0.29	0.76	0.35	0.53	0.30	0.98	0.43
RW	0.51	0.42	1.02	0.56	0.50	0.28	0.74	0.30	0.59	0.29	0.91	0.41
D3	0.63	0.38	1.15	0.51	0.50	0.25	0.80	0.34	0.50	0.27	0.94	0.40
DMPO	0.76	0.26	1.54	0.43	0.44	0.20	0.81	0.29	0.19	0.10	0.39	0.15

Table 1: The performance (%) comparison between the baselines and CausalDPO on the three datasets with three OOD distributions . We highlight the methods with the best and second-best results.

Figure 2: Further study on performance under distribution shifts and clustering Visualization.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Movielens-10M	Yelp2018	Book-Crossing
#Sequence	71,567	31,668	278,858
#Items	10,681	38,048	271,379
#Interactions	10,000,054	1,561,406	1,149,780

Appendix

Table 2: Statistics of datasets.

Methods	ML-10M				Book-Crossing
Methods	H@10	N@10	H@20	N@20	H@10	N@10	H@20	N@20
SASRec	0.47	0.17	0.75	0.27	1.70	0.90	3.12	1.02
BIGRec	0.66	0.31	1.00	0.40	1.64	0.73	2.19	0.87
RW	0.77	0.45	1.03	0.45	1.52	0.81	1.89	0.95
$D^{3}$	0.72	0.41	0.99	0.40	1.48	0.71	2.03	0.82
DMPO	0.63	0.33	0.97	0.42	1.61	0.79	2.25	0.91

Appendix

Table 3: The performance (%) comparison between the baselines and CausalDPO on the two datasets with three IID distributions . We highlight the methods with the best and second-best performances.

Metric	SASRec	BIGRec	RW	$D^{3}$	DMPO	SDPO	RosePO	SPRec	Ours
H@10	0.48	0.49	0.25	0.45	0.50	0.51	0.39	0.41	0.53
N@10	0.20	0.22	0.08	0.18	0.14	0.15	0.19	0.20	0.24
H@20	0.72	0.76	0.51	0.68	0.75	0.76	0.70	0.75	1.51
N@20	0.25	0.28	0.15	0.23	0.20	0.21	0.26	0.28	0.47

Appendix

Table 4: Model Performance (%) Comparison Across Two Distribution Shift Types. We highlight the methods with the best and second-best performances.

Methods	ML-10M				Book-Crossing
Methods	H@10	N@10	H@20	N@20	H@10	N@10	H@20	N@20
Dr.DPO	0.17	0.20	0.55	0.18	0.39	0.19	0.50	0.26
CPO	0.41	0.25	0.81	0.31	0.34	0.20	0.54	0.24
SimPO	0.29	0.21	0.58	0.23	0.39	0.18	0.61	0.24
DPO	0.43	0.20	0.81	0.29	0.19	0.10	0.39	0.20
SimPO w/ Causal Reg.	0.36	0.27	0.64	0.34	0.45	0.03	0.70	0.30

Appendix

Table 5: Comparison of models with and without causal regularization on ML-10M and Book.

Methods	DPO	CPO	SimPO	CausalDPO
Complexity	$\mathcal{O}(3B\cdot L^{2}\cdot d)$	$\mathcal{O}(3B\cdot L^{2}\cdot d+BnLd)$	$\mathcal{O}(2B\cdot L^{2}\cdot d)$	$\mathcal{O}(B\cdot L^{2}\cdot d+B^{2})$
seconds / epoch	2482s	2160s	1445s	2971s

Appendix

Table 6: Computational Complexity Comparison of Models

Methods	Dr.DPO w/ Causal	CPO w/ Causal	SimPO w/ Causal	CausalDPO
seconds / epoch	2504s	2659s	2280s	2971s

Appendix

Table 7: Computational Complexity Comparison of Models with Causal Invariant Regularization

Variant	Yelp2018		ML-10M		Book-Crossing
Variant	H@10	N@10	H@10	N@10	H@10	N@10
w/o SFT	1.28	0.54	0.52	0.26	0.25	0.12
w/o CausalDPO	1.47	0.69	0.55	0.28	0.52	0.30
w/o Both	1.32	0.37	0.46	0.22	0.27	0.12
Ours	1.53	0.77	0.58	0.32	0.64	0.37

Appendix

Table 8: Ablation study results (%).

Methods	Yelp2018				ML-10M
Methods	H@10	N@10	H@20	N@20	H@10	N@10	H@20	N@20
SASRec	1.01	0.43	1.51	0.61	0.50	0.02	0.84	0.30
BIGRec	0.77	0.50	1.28	0.62	0.53	0.29	0.76	0.35
RW	0.51	0.42	1.02	0.56	0.50	0.28	0.74	0.30
D3	0.63	0.38	1.15	0.51	0.50	0.25	0.80	0.34
Ours	1.53	0.77	1.79	0.82	0.61	0.35	1.02	0.45

Appendix

Table 9: Under two OOD distribution settings on two datasets, we compare the performance (%) of CausalDPO with different model sizes against strong baselines. The best results are highlighted in red , and the second-best results are highlighted in blue .

Figure 3: Evaluating the impact of model hyperparameter sets on recommendation performance.

Large Language Model-Based Recommendation

Large language model-based recommendation uses language models to personalize ranking, retrieval, and conversational suggestions from text and behavior.

See all 207 papers

TPO: Tri-level Distributionally Robust Learning for OOD Direct Preference Optimization
Oct 20, 2026Chengtao Jian, Kai YangLLM Alignment Direct Preference Optimization
School of Computer Science and Technology, Tongji University, Shanghai, China · Shenzhen Loop Area Institute, Shenzhen, China · Shanghai Research Institute for Intelligent Autonomous Systems, Shanghai, China
Causal Representation Learning for Generalisable Recommendation
May 26, 2026Yorgos Felekis, Michael O'Riordan, Oriol Corcoll +1Disentangled Representation Learning Causal Representation Learning
University of Warwick · Spotify · Spotify & University College London
Conditionally Identifiable Latent-Environment Modeling for Out-of-Distribution Recommendation
Aug 4, 2026Qianqian Wang, Wenwu Gong, Yunshan Li +3Latent Variable Models OOD Generalization
Shenzhen Key Laboratory of Safety and Security for Next Generation of Industrial Internet, Southern University of Science and Technology, Shenzhen, China · Department of Statistics and Data Science, Southern University of Science and Technology, Shenzhen, China · School of Mathematical and Computational Sciences, Massey University, Auckland, New Zealand

Oct 20, 2026J/K move · Enter open · S save

TPO: Tri-level Distributionally Robust Learning for OOD Direct Preference Optimization

Chengtao Jian, Kai Yang

School of Computer Science and Technology, Tongji University, Shanghai, China · Shenzhen Loop Area Institute, Shenzhen, China · Shanghai Research Institute for Intelligent Autonomous Systems, Shanghai, China

LLM Alignment Direct Preference Optimization Distribution Shift OOD Generalization Distributionally Robust Optimization Group Distributionally Robust Optimization

Open paper AskPDF arXiv

Direct Preference Optimization (DPO) minimizes empirical risk over observed preference comparisons, implicitly assuming that the training preference distribution matches the deployment preference distribution. In practice, this i.i.d. assumption often fails, as preference distribution shift exhibits a natural hierarchical structure across three levels: group-level shift, prompt-level shift, and preference-level shift. Such multi-level distributional mismatch poses a critical challenge for safe LLM deployment, yet existing robust DPO methods typically either model uncertainty over the joint preference distribution or focus on individual sources of shift, leaving the hierarchical structure of OOD preference shifts underexplored. We propose TPO, a tri-level distributionally robust framework for OOD preference alignment, which jointly models all three levels of distributional uncertainty through a hierarchically structured rectangular ambiguity set. By instantiating it with Wasserstein distance, we derive a tractable and interpretable objective that combines a closed-form preference-level margin penalty, prompt-level adversarial embedding perturbation, and worst-case group reweighting, each governed by an independent radius. We establish finite-sample convergence guarantees and empirically validate the effectiveness of TPO over existing robust DPO methods.

Abstract

Large Language Model-Based Recommendation

Explore similar work

TPO: Tri-level Distributionally Robust Learning for OOD Direct Preference Optimization

Causal Representation Learning for Generalisable Recommendation

Conditionally Identifiable Latent-Environment Modeling for Out-of-Distribution Recommendation