Proxy-Guided Measurement Calibration

Saketh Vishnubhatla; Shu Wan; Andre Harrison; Adrienne Raglin; Huan Liu

cs.LGMar 10, 2026

Proxy-Guided Measurement Calibration

Authors: Saketh Vishnubhatla, Shu Wan, Andre Harrison, Adrienne Raglin, Huan Liu

Organizations: Arizona State University, Tempe, AZ · DEVCOM Army Research Lab, Adelphi, MD

Abstract

Aggregate outcome variables collected through surveys and administrative records are often subject to systematic measurement error. For instance, in disaster loss databases, county-level losses reported may differ from the true damages due to variations in on-the-ground data collection capacity, reporting practices, and event characteristics. Such miscalibration complicates downstream analysis and decision-making. We study the problem of outcome miscalibration and propose a framework guided by proxy variables for estimating and correcting the systematic errors. We model the data-generating process using a causal graph that separates latent content variables driving the true outcome from the latent bias variables that induce systematic errors. The key insight is that proxy variables that depend on the true outcome but are independent of the bias mechanism provide identifying information for quantifying the bias. Leveraging this structure, we introduce a two-stage approach that utilizes variational autoencoders to disentangle content and bias latents, enabling us to estimate the effect of bias on the outcome of interest. We analyze the assumptions underlying our approach and evaluate it on synthetic data, semi-synthetic datasets derived from randomized trials, and a real-world case study of disaster loss reporting.

Figures & tables

Panel of Figure 1 — Figure 1: Causal structure for proxy-guided measurement calibration. (a) Full generative model: environment variables $E$ influence latent content factors $Z$ and latent reporting bias $A$ . Proxy measurements $\{Y_{1},\ldots,Y_{m}\}$ depend only on $Z$ , while the observed outcome $Y_{\mathrm{obs}}$ depends on both $Z$ and $A$ . (b) Error model highlighting the measurement bias mechanism: the true outcome $Y_{\mathrm{true}}$ is perturbed by an environment-dependent binary bias $A$ with additive magnitude $\alpha$ .

	Synthetic	Semi-synthetic	Real-world
(A1) Proxy exclusion	$\checkmark$	$\checkmark$	$\checkmark$
(A2) Bias model: constant $\alpha$	$\checkmark$	$\checkmark$	$\times$
(A3) Sign restriction: $\alpha\geq 0$	$\checkmark$	$\checkmark$	$\times$
(A4) Linear structural mechanisms	$\checkmark$	$\times$	$\times$
(A5) Availability of ground-truth $(A,Z)$	$\checkmark$	$\circ$	$\times$
(A6) Overlap in $A$	$\checkmark$	$\checkmark$	$\circ$

Table 1: Assumptions across experimental settings.

\checkmark

indicates the assumption is satisfied,

\times

indicates it is not assumed, and

\circ

indicates partial or indirect availability.

			Estimated $\alpha$
			$\alpha=1$		$\alpha=5$		$\alpha=10$
$n$	$d_{z}$	$d_{e}$	Gaussian	Poisson (scaled)	Gaussian	Poisson (scaled)	Gaussian	Poisson (scaled)
500	1	10	1.20±0.23	1.26±0.23	4.16±0.65	4.17±0.64	8.59±1.48	8.61±1.47
500	2	10	1.07±0.28	1.06±0.36	3.88±0.51	3.63±0.48	8.23±1.47	7.82±1.65
500	5	10	1.08±0.22	0.88±0.33	3.95±0.48	3.71±0.65	8.61±0.93	8.19±0.93
500	10	10	1.01±0.28	1.06±0.15	3.81±1.37	4.03±1.07	7.58±2.31	8.16±1.56

Table 2: Bias estimation with synthetic data under different settings.

	JOBS			OHIE
	$\alpha=1$	$\alpha=5$	$\alpha=10$	$\alpha=1$	$\alpha=5$	$\alpha=10$
Baseline (Proxy only)	4.718 ± 0.03	8.382 ± 0.04	13.382 ± 0.04	22.418 ± 0.05	24.073 ± 0.06	27.973 ± 0.07
Baseline (Env only)	3.526 ± 0.13	7.010 ± 0.26	11.979 ± 0.26	15.204 ± 0.55	15.200 ± 0.47	16.114 ± 0.29
Baseline (TEDVAE)	0.505 ± 0.32	1.304 ± 0.71	2.275 ± 1.37	0.225 ± 0.18	0.279 ± 0.18	0.769 ± 0.43
Ours dim. (Z=5)	1.694 ± 0.42	3.807 ± 1.14	7.580 ± 1.30	1.807 ± 0.37	4.415 ± 0.30	7.700 ± 0.36
Ours dim. (Z=10)	1.505 ± 0.35	3.529 ± 0.89	6.518 ± 1.63	1.941 ± 0.36	4.320 ± 0.18	7.548 ± 0.28

Table 3: Estimated

\hat{\alpha}

for JOBS and OHIE across baselines and latent dimensions.

Panel of Figure 2 — Figure 2: County-level mean absolute CATE estimates $|\widehat{\tau}_{i}|$ for 2023 across four hazard types. White indicates counties with no event; lighter colors represent higher estimated reporting bias.

Figure 3: Mean absolute CATE values grouped by hazard type. Wildfire events exhibit the highest systematic reporting distortion, followed by tornadic and flooding events.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Scatter plot of the recovered content latents $\hat{Z}$ versus the true latents $Z$ for a representative fold, after aligning dimensions using the closest permutation that maximizes permuted $R^{2}$ .

Figure 5: Sanity Check for the Distribution. Histograms for all key variables:

E_{1}

Z

A

Y_{\text{true}}

Y_{\text{obs}}

, and proxies for a given configuration. Ensures data follows the intended generative model.

Figure 6: Sanity Check - Core Metrics and Correlations. This page summarizes the main structural sanity metrics: $R^{2}(Y_{\text{true}}\mid Z)$ , $R^{2}(Y_{\text{obs}}\mid Y_{\text{true}})$ , the effect of bias latents via $\Delta R^{2}$ , per-dimension $R^{2}$ for each $Z_{i}$ , proxy alignment metrics, as well as the correlation matrix over $\{Z,A,Y\}$ . These diagnostics confirm that the dataset correctly encodes both latent content and latent bias structure.

			$a_{mae}$
			$\alpha=1$		$\alpha=5$		$\alpha=10$
$n$	$d_{z}$	$d_{e}$	Gaussian	Poisson (scaled)	Gaussian	Poisson (scaled)	Gaussian	Poisson (scaled)
500	1	10	1.29±0.33	1.29±0.32	1.39±0.47	1.38±0.49	1.48±0.50	1.46±0.51
500	2	10	1.52±0.22	1.54±0.21	1.63±0.34	1.64±0.34	1.81±0.39	1.85±0.43
500	5	10	1.37±0.27	1.29±0.38	1.09±0.31	1.18±0.28	1.19±0.29	1.13±0.37
500	10	10	1.68±0.34	1.68±0.35	1.72±0.47	1.82±0.40	1.83±0.41	1.83±0.44

Appendix

Table 4: Mean

\pm

Std of

a_{mae}

grouped by

(n,d_{z},d_{e})

rows and

(\alpha,\text{noise})

columns.

			$rmse_{proxies}$
			$\alpha=1$		$\alpha=5$		$\alpha=10$
$n$	$d_{z}$	$d_{e}$	Gaussian	Poisson (scaled)	Gaussian	Poisson (scaled)	Gaussian	Poisson (scaled)
500	1	10	0.20±0.03	0.20±0.03	0.20±0.03	0.20±0.03	0.20±0.03	0.20±0.03
500	2	10	0.21±0.03	0.20±0.02	0.21±0.03	0.20±0.02	0.21±0.03	0.20±0.02
500	5	10	0.19±0.02	0.19±0.02	0.19±0.02	0.19±0.02	0.19±0.02	0.19±0.02
500	10	10	0.15±0.02	0.15±0.02	0.15±0.02	0.15±0.02	0.15±0.02	0.15±0.02

Appendix

Table 5: Mean

\pm

Std of

rmse_{proxies}

grouped by

(n,d_{z},d_{e})

rows and

(\alpha,\text{noise})

columns.

			$z_{mae}$
			$\alpha=1$		$\alpha=5$		$\alpha=10$
$n$	$d_{z}$	$d_{e}$	Gaussian	Poisson (scaled)	Gaussian	Poisson (scaled)	Gaussian	Poisson (scaled)
500	1	10	1.84±0.86	1.82±0.86	1.84±0.86	1.82±0.86	1.84±0.86	1.82±0.86
500	2	10	1.42±0.41	1.50±0.37	1.42±0.41	1.50±0.37	1.42±0.41	1.50±0.37
500	5	10	1.22±0.19	1.23±0.19	1.22±0.19	1.23±0.19	1.22±0.19	1.23±0.19
500	10	10	1.14±0.06	1.14±0.07	1.14±0.06	1.14±0.07	1.14±0.06	1.14±0.07

Appendix

Table 6: Mean

\pm

Std of

z_{mae}

grouped by

(n,d_{z},d_{e})

rows and

(\alpha,\text{noise})

columns.

Variable Type	Name(s)	Description
Outcome	$Y_{\mathrm{obs}}$	Log-transformed emergency department spending ( $0$ – $11.13$ ).
Proxies	$Y_{\mathrm{proxy},1}$ – $Y_{\mathrm{proxy},5}$	Biomedical measurements: systolic BP, diastolic BP, HDL, A1C, PHQ depression score.
Environment $E$ : Treatment	$E_{1}$ (T)	Lottery selection indicator (0/1).
Environment $E$ : State program participation	$E_{2}$ : SNAP household (prenotify) $E_{3}$ : TANF household (prenotify)	SNAP/TANF participation prior to randomization (large variation: 286–2952 unique values).
Environment $E$ : Prior ED utilization	$E_{4}$ : num_visit_pre_cens_ed $E_{5}$ : num_on_pre_cens_ed $E_{6}$ : num_off_pre_cens_ed $E_{7}$ : num_chron_pre_cens_ed $E_{8}$ : num_inj_pre_cens_ed $E_{9}$ : num_hiun_pre_cens_ed $E_{10}$ : num_loun_pre_cens_ed $E_{11}$ : ed_charg_tot_pre_ed $E_{12}$ : charg_tot_pre_ed	Utilization counts and total pre-period ED charges. Reflects burden of chronic, injury-related, heart, head, and psychiatric emergency visits. Many exhibit wide ranges (e.g., $0–$ 180,054 in total charges).
Environment $E$ : In-person clinical measures	$E_{13}$ : tot_med_spend_other_inp	Supplementary in-person medical expenditure measure (0–92,475 across subjects).

Appendix

Table 7: Schema of the OHIE semi-synthetic dataset after preprocessing. Environment variables

E

include treatment, state-program participation, prior emergency-department utilization, and in-person clinical measures. Proxies are clean biomedical measurements and the outcome is log-transformed ED spending.

Variable Type	Name(s)	Description
Outcome	$Y_{\mathrm{obs}}$	Log-transformed 1978 earnings ( $\log(1+\mathrm{re78})$ ), range $0$ – $11.01$ , with 457 unique values.
Proxies	$Y_{\mathrm{proxy},1}$ , $Y_{\mathrm{proxy},2}$	Clean pretreatment earnings: • $Y_{\mathrm{proxy},1}$ (re74): range $0$ – $35{,}040$ , 358 unique values • $Y_{\mathrm{proxy},2}$ (re75): range $0$ – $25{,}142$ , 356 unique values These proxies are unaffected by the injected measurement bias.
Environment $E$ : Treatment	$E_{1}$ (treat)	Random assignment to the job-training intervention (binary 0/1).
Environment $E$ : Demographics	$E_{2}$ : age (16–55; 40 unique values) $E_{3}$ : educ (0–18; 19 unique values) $E_{4}$ : black (0/1) $E_{5}$ : hispan (0/1) $E_{6}$ : married (0/1) $E_{7}$ : nodegree (0/1)	Baseline demographic variables from the NSW/JOBS dataset, representing socioeconomic and educational background.
Bias latent	$A$	Synthetic binary latent generated from a logistic function of the environment variables, used to inject controlled bias into the observed outcome.

Appendix

Table 8: Schema of the JOBS semi-synthetic dataset after preprocessing. Environment variables consist of treatment and demographic covariates; proxies are clean pretreatment earnings; the outcome is log-transformed 1978 earnings.

Category	Variables and description
Identifiers	FIPS (county identifier), Year, Month
Hazard indicators	Hazard type (Flooding, Hurricane/Tropical Storm, Tornado, Wildfire), one-hot encoded as $E_{\text{hazard},k}$
Observed outcome	$Y_{\mathrm{obs}}$ : reported county-level economic loss (property damage, inflation-adjusted to 2023 USD), log-transformed and standardized
Proxy variables	$Y_{\mathrm{proxy},1}$ : fraction of land transitioning from bare ground to water
	$Y_{\mathrm{proxy},2}$ : fraction of built area transitioning to damage-related classes
	$Y_{\mathrm{proxy},3}$ : fraction of built area transitioning to water

Appendix

Table 9: Variables used in the SHELDUS-based real-world case study after preprocessing.

Proximal Causal Inference

See all 9 papers

May 7, 2026

Estimate Level Adjustment For Inference With Proxies Under Random Distribution Shifts

In many scientific domains, including experimentation, researchers rely on measurements of proxy outcomes to achieve faster and more frequent reads, especially when the primary outcome of interest is challenging to measure directly. While proxies offer a more readily accessible observation for inference, the ultimate goal is to draw statistical inferences about the primary outcome parameter and proxy data are typically imperfect in some ways. To correct for these imperfections, current statistical inference methods often depend on strict identifying assumptions (such as surrogacy, covariate/label shift, or missingness assumptions). These assumptions can be difficult to validate and may be violated by various additional sources of distribution shift, potentially leading to biased parameter estimates and miscalibrated uncertainty quantification. We introduce an estimate-level framework, inspired by domain adaptation techniques, to empirically calibrate proxy-based inference. This framework models the proxy-primary metric discrepancy as a random effect at the parameter level, estimating its distribution from aggregated historical observations across past domains (e.g., experiments, time periods, or distinct segments). This method avoids the requirement for retaining individual-level response data. Additionally, this adjustment can be layered on top of existing proxy-correction methods (such as prediction-powered inference or importance weighting) to account for additional biases not addressed by those corrections. To manage uncertainty when the number of historical domains is limited, we provide both a method-of-moments estimator and a domain bootstrap procedure. We further validate this approach using publicly available datasets and real-world experiments.

Steven Wilkins-Reeves, Alexandra N. M. Darmon, Deeksha Sinha

Central Applied Science, Meta · New York, NY, USA · Menlo Park, CA, USA

Distribution Shift Robustness Uncertainty Calibration

Ask PDF

Jul 27, 2026

proxymate: Diagnosis and Adjustment of Proxy Estimates for Reliable Inference

Proxy outcomes (such as short-term behavioral signals, model predictions, or surrogate endpoints) are frequently used in place of primary outcomes that are too slow to mature, rare, or challenging to measure directly. But valid inference on a proxy does not guarantee valid inference on the primary estimate as proxy-based estimates can be systematically biased in ways that are difficult to predict, leading to improperly calibrated confidence intervals. We present proxymate, a framework and open-source Python package for proxy validation and adjustment. proxymate organizes into four levels: The Representativity Level (population validity), the Unit Level (measurement quality), the Estimate Level (decision validity), and the Domain Level (cross-domain transportability). Within each level, proxymate provides diagnostic checks, and targeted adjustment strategies that map specific failures to appropriate corrections. At Meta, proxymate has been adopted by many different use cases, spanning experimentation, prevalence estimation, and monitoring use cases, all facing different proxy challenges (limited human review time, long maturation window of outcomes, low detectability) and showcasing the modularity of the framework. Across all products, proxymate assessed and corrected millions of proxy, primary unit comparisons. It has facilitated launches across multiple work streams including enabling quick decision making on thousands of experiments.

Alexandra N. M. Darmon, Deeksha Sinha, Steve Wilkins-Reeves +1

Proximal Balancing for Causal Effect Estimation under Unmeasured Confounding

Estimating causal effects from observational data is central to science and policy, but the effects are not identified when confounders are unmeasured. Proximal causal inference addresses this problem with proxies of the unmeasured confounders. However, existing proxy-based approaches either designate proxy roles and solve an inverse problem, which is ill-posed and hard to estimate with high-dimensional proxies, or use a latent-variable model, which assumes that the learned latent variable matches the hidden confounder and leaves bias when it does not. To address these challenges, we introduce proximal balancing. It carries the classical idea of covariate balancing to confounders that are observed only through proxies: it learns a low-dimensional summary of the covariates and proxies that makes the treatment groups comparable, and then adjusts for this summary. It needs no designated proxy roles, inverse problem, or latent model. We give identification theory, finite-sample guarantees, and a practical algorithm, PROBE. We demonstrate the method on low-dimensional, high-dimensional, and image proxies and on real-world data.

Yonghan Jung

School of Information Sciences University of Illinois Urbana-Champaign

Causal Effect Estimation Proximal Causal Inference Causal Inference Causal Inference with Unobserved Confounding

Ask PDF

Abstract

Proximal Causal Inference

Explore similar work

Estimate Level Adjustment For Inference With Proxies Under Random Distribution Shifts

proxymate: Diagnosis and Adjustment of Proxy Estimates for Reliable Inference

Proximal Balancing for Causal Effect Estimation under Unmeasured Confounding