cs.LGMar 23, 2026

Off-Policy Evaluation for Ranking Policies under Deterministic Logging Policies

Authors: Koichi Tanaka, Kazuki Kawamura, Takanori Muroi, Yusuke Narita, Yuki Sasamoto, Kei Tateno, Takuma Udagawa, Wei-Wei Du, 

Organizations: Keio University · Sony Group Corporation · Yale University · Hanuku-kaso, Co., Ltd.

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Off-Policy Evaluation (OPE) is an important practical problem in algorithmic ranking systems, where the goal is to estimate the expected performance of a new ranking policy using only offline logged data collected under a different, logging policy. Existing estimators, such as the ranking-wise and position-wise inverse propensity score (IPS) estimators, require the data collection policy to be sufficiently stochastic and suffer from severe bias when the logging policy is fully deterministic. In this paper, we propose novel estimators, Click-based Inverse Propensity Score (CIPS), exploiting the intrinsic stochasticity of user click behavior to address this challenge. Unlike existing methods that rely on the stochasticity of the logging policy, our approach uses click probability as a new form of importance weighting, enabling low-bias OPE even under deterministic logging policies where existing methods incur substantial bias. We provide theoretical analyses of the bias and variance properties of the proposed estimators and show, through synthetic and real-world experiments, that our estimators achieve significantly lower bias compared to strong baselines, for a range of experimental settings with completely deterministic logging policies.

Figures & tables

positionactionobservable reward CRCRclick CCpotential reward RR
k=1k=1a1a_{1}000030003000
k=2k=2a2a_{2}100010001110001000
k=3k=3a3a_{3}000020002000
k=4k=4a4a_{4}0000500500
Table 1: Examples of clicks, potential rewards, and observable rewards.
Table 2: A toy example of importance weight under completely deterministic logging policy.
Table 3: A toy example of click importance weight under completely deterministic logging policy. The new and logging policy are the same as the previous example given in Table 2(c) .
Figure 1: MSE, Squared Bias, and Variance with varying logged data sizes ( nn ).
Figure 5: MSE, Squared Bias, and Variance with varying logged data sizes ( nn ).
Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Table 4: A toy example of importance weight under completely deterministic logging policy.
Figure 7: Comparison of MSE, Squared Bias, and Variance with varying new policies ( ϵ\epsilon ).
Figure 8: Comparison of MSE, Squared Bias, and Variance with different policy definition.
Figure 9: Comparison of MSE, Squared Bias, and Variance with varying estimation noise on pcp_{c} .
Figure 10: Comparison of CIPS and CDR’ MSE, Squared Bias, and Variance with varying estimation noises on qrq_{r} .
Figure 11: Comparison of the policy selection accuracy with varying logged data sizes ( nn ) and deterministic user thresholds ( α\alpha ).
More in this topic

Off-Policy Evaluation

Off-policy evaluation estimates a policy’s value from data collected by another policy, using weighting, models, and uncertainty-aware methods.

See all 72 papers

Explore similar work

CardsList

  1. Adaptive Doubly Robust Off-Policy Evaluation for Ranking Policies under Diverse User Behavior

    Aug 30, 2026Kosuke Iguchi, Ren KishimotoOff-Policy EvaluationDoubly Robust Estimation

    Institute of Science Tokyo Tokyo, Japan

  2. Recommendation Ranking Off-Policy Evaluation under Ranking-Dependent Examination via Examination-Relevance Decomposition

    Sep 28, 2026Riki Okamura, Toshiharu SugawaraOff-Policy EvaluationDoubly Robust Estimation

    Department of Computer Science and Communications Engineering, Waseda University, Tokyo, Japan

  3. Efficient Offline Learning of Ranking Policies via Top-kk Policy Decomposition

    Sep 29, 2026Ren Kishimoto, Koichi Tanaka, Haruka Kiyohara +4Learning to RankOff-Policy RL

    Hanjuku-kaso, Co., Ltd. Tokyo, Japan