Performative Prediction with Selective Labels

Giovani Valdrighi; Isabel Valera; Marcos Medeiros Raimundo

cs.LGOct 6, 2026

Performative Prediction with Selective Labels

Authors: Giovani Valdrighi, Isabel Valera, Marcos Medeiros Raimundo

Organizations: Instituto de Computação Universidade Estadual de Campinas Campinas, Brazil · Department of Computer Science Saarland University Saarbrücken, Germany

Abstract

Many social applications of machine learning exhibit performative effects: population behavior changes in response to deployed models. Performative prediction studies this interaction through a distribution map that relates each model to the population distribution it induces. One of the main results in this framework showed that repeated risk minimization (RRM), which updates models by retraining on the most recent data, can converge to a stable model that minimizes risk on its own induced distribution. However, existing analyses typically assume access to the complete distributions of features and labels after model deployment, ignoring the possibility of selective labels: observing labels only for the accepted subset of the population. In this work, we formalize performative prediction with selective labels and show that retraining only on observed data can misguide the retraining procedure and undermine the guarantees of convergence to a stable solution. We then propose a worst-case objective based on knowledge of a confidence interval on the probability of a positive label. Applying RRM to this objective permits us to remain within a bounded distance to the true stable point. Under a sensitivity assumption on the conditional label distribution, we further show how previously accepted data can tighten these confidence intervals over time. Experiments in a lending application with fairness regularization show that our robust optimization approach closely matches the performance of RRM with complete label access.

Figures & tables

Panel of Figure 1 — Figure 1 : Observation dynamics for performative prediction with selective labels.

Panel of Figure 2 — Figure 2 : Evaluation of Robust RRM. (a) Performative gap across different sensitivity levels of the distribution map (averages and 90% interval over 10 random seeds). (b) Comparison of executing Robust RRM with trivial limits versus learned limits (averages and standard deviations over 10 random seeds).

Figure 3 : Loss, disparity, and distance to the stable solution for RRM with different objective configurations using

\epsilon=50

. The y-axis was cropped to improve readability and does not display the full results for the DR Estimator agent. Results are averages and standard deviations over 10 random seeds.

Constant used (%misspecification)	Performative risk $(\downarrow)$	Dist. to $\theta^{PS}$ $(\downarrow)$
$L_{x}=0.15$ correctly specified	$0.336(\pm 0.001)$	$0.212(\pm 0.025)$
$L_{x}+0.0375$ (+25%)	$0.336(\pm 0.001)$	$0.241(\pm 0.020)$
$L_{x}-0.0375$ (-25%)	$0.336(\pm 0.001)$	$0.187(\pm 0.018)$
$L_{x}+0.075$ (+50%)	$0.336(\pm 0.001)$	$0.282(\pm 0.028)$
$L_{x}-0.075$ (-50%)	$0.336(\pm 0.001)$	$0.159(\pm 0.024)$

Table 1 : Results of Robust RRM with misspecified

L_{x}

. Results are averages (and standard deviations) over 10 random seeds. The distance to the stable solution reaches a maximum of

0.282

with +50% misspecification.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Number of samples	$\widehat{L}_{x}$ (and std. over 10 random seeds)
200	$0.457(\pm 0.148)$
1000	$0.331(\pm 0.129)$
2000	$0.287(\pm 0.062)$
10000	$0.207(\pm 0.034)$

Appendix

Table 2 : Estimation of constant

L_{x}

under the assumption of a logistic label distribution and randomized acceptance, independently of model decision.

Figure 4 : Distribution of scores and positive label calculated from the FICO dataset.

z=1

represents individuals who self-identify as Black and

z=0

individuals who self-identify as white.

Figure 5 : Loss, disparity, and distance to the stable solution presented by RRM with different objective configurations using

\epsilon=90

. Results are averages (and std.) over 10 random seeds.

Figure 6 : Loss, disparity, and distance to the stable solution for RRM with different objective configurations using

\epsilon=150

. Results are averages (and std.) over 10 random seeds.

Figure 7 : Loss, disparity, and distance to the stable solution in the environment with outcome performativity with

\epsilon=50

. The y-axis was cropped to improve readability and does not display the full results for the DR Estimator agent. Results are averages (and std.) over 10 random seeds.

Explore similar work

Partially Performative Prediction
Jun 5, 2026Jaewook Lee, Tijana ZrnicDistribution Shifts Feedback Loop
Department of Electrical Engineering, Stanford University · Departments of Statistics and Management Science & Engineering, Stanford University
Beyond the Training Distribution: Evaluating Predictions Under Distribution Shift and Selection Bias
Jun 12, 2026Annie Ulichney, Amanda CostonCovariate Shift Selection Bias
Department of Statistics, University of California, Berkeley
Retraining Seeks Stable Signals
Jul 17, 2026Moritz HardtRetraining Heterogeneous Treatment Effects
Max Planck Institute for Intelligent Systems, Tübingen, and Tübingen AI Center

Jun 5, 2026, cs.LGJ/K move · Enter open · S save

Partially Performative Prediction

Jaewook Lee, Tijana Zrnic

Department of Electrical Engineering, Stanford University · Departments of Statistics and Management Science & Engineering, Stanford University

Distribution Shifts Feedback Loop Heuristics

Open paper AskPDF arXiv

Performative prediction studies feedback loops that arise when predictive models are deployed in consequential domains. In these settings, deploying a model can change the population whose patterns the model aims to predict, inducing a distribution shift that is endogenous to the learning system. This perspective departs from classical treatments of distribution shift, where shifts are typically modeled as exogenous changes in the data-generating process. Yet, in practice, distribution shift is rarely one or the other. Predictive models may influence future data through the decisions they support, while the world itself continues to drift for reasons beyond the learner's control. We study partially performative prediction, a framework that captures both endogenous and exogenous sources of distribution shift. The framework generalizes performative prediction by allowing the data distribution to evolve both in response to the deployed model and according to an external, time-varying process. We extend the central notions of performative stability and performative optimality to this setting by defining their online analogues that track the evolving partially performative environment. We analyze practical learning heuristics, including repeated retraining, and characterize when they successfully adapt to partially performative environments.

Abstract

Explore similar work

Partially Performative Prediction

Beyond the Training Distribution: Evaluating Predictions Under Distribution Shift and Selection Bias

Retraining Seeks Stable Signals