Proxy-Guided Measurement Calibration

Saketh Vishnubhatla; Shu Wan; Andre Harrison; Adrienne Raglin; Huan Liu

cs.LGMar 10, 2026

Proxy-Guided Measurement Calibration

Authors: Saketh Vishnubhatla, Shu Wan, Andre Harrison, Adrienne Raglin, Huan Liu

Organizations: Arizona State University, Tempe, AZ · DEVCOM Army Research Lab, Adelphi, MD

Abstract

Aggregate outcome variables collected through surveys and administrative records are often subject to systematic measurement error. For instance, in disaster loss databases, county-level losses reported may differ from the true damages due to variations in on-the-ground data collection capacity, reporting practices, and event characteristics. Such miscalibration complicates downstream analysis and decision-making. We study the problem of outcome miscalibration and propose a framework guided by proxy variables for estimating and correcting the systematic errors. We model the data-generating process using a causal graph that separates latent content variables driving the true outcome from the latent bias variables that induce systematic errors. The key insight is that proxy variables that depend on the true outcome but are independent of the bias mechanism provide identifying information for quantifying the bias. Leveraging this structure, we introduce a two-stage approach that utilizes variational autoencoders to disentangle content and bias latents, enabling us to estimate the effect of bias on the outcome of interest. We analyze the assumptions underlying our approach and evaluate it on synthetic data, semi-synthetic datasets derived from randomized trials, and a real-world case study of disaster loss reporting.

Figures & tables

Panel of Figure 1 — Figure 1: Causal structure for proxy-guided measurement calibration. (a) Full generative model: environment variables $E$ influence latent content factors $Z$ and latent reporting bias $A$ . Proxy measurements $\{Y_{1},\ldots,Y_{m}\}$ depend only on $Z$ , while the observed outcome $Y_{\mathrm{obs}}$ depends on both $Z$ and $A$ . (b) Error model highlighting the measurement bias mechanism: the true outcome $Y_{\mathrm{true}}$ is perturbed by an environment-dependent binary bias $A$ with additive magnitude $\alpha$ .

	Synthetic	Semi-synthetic	Real-world
(A1) Proxy exclusion	$\checkmark$	$\checkmark$	$\checkmark$
(A2) Bias model: constant $\alpha$	$\checkmark$	$\checkmark$	$\times$
(A3) Sign restriction: $\alpha\geq 0$	$\checkmark$	$\checkmark$	$\times$
(A4) Linear structural mechanisms	$\checkmark$	$\times$	$\times$
(A5) Availability of ground-truth $(A,Z)$	$\checkmark$	$\circ$	$\times$
(A6) Overlap in $A$	$\checkmark$	$\checkmark$	$\circ$

Table 1: Assumptions across experimental settings.

\checkmark

indicates the assumption is satisfied,

\times

indicates it is not assumed, and

\circ

indicates partial or indirect availability.

			Estimated $\alpha$
			$\alpha=1$		$\alpha=5$		$\alpha=10$
$n$	$d_{z}$	$d_{e}$	Gaussian	Poisson (scaled)	Gaussian	Poisson (scaled)	Gaussian	Poisson (scaled)
500	1	10	1.20±0.23	1.26±0.23	4.16±0.65	4.17±0.64	8.59±1.48	8.61±1.47
500	2	10	1.07±0.28	1.06±0.36	3.88±0.51	3.63±0.48	8.23±1.47	7.82±1.65
500	5	10	1.08±0.22	0.88±0.33	3.95±0.48	3.71±0.65	8.61±0.93	8.19±0.93
500	10	10	1.01±0.28	1.06±0.15	3.81±1.37	4.03±1.07	7.58±2.31	8.16±1.56

Table 2: Bias estimation with synthetic data under different settings.

	JOBS			OHIE
	$\alpha=1$	$\alpha=5$	$\alpha=10$	$\alpha=1$	$\alpha=5$	$\alpha=10$
Baseline (Proxy only)	4.718 ± 0.03	8.382 ± 0.04	13.382 ± 0.04	22.418 ± 0.05	24.073 ± 0.06	27.973 ± 0.07
Baseline (Env only)	3.526 ± 0.13	7.010 ± 0.26	11.979 ± 0.26	15.204 ± 0.55	15.200 ± 0.47	16.114 ± 0.29
Baseline (TEDVAE)	0.505 ± 0.32	1.304 ± 0.71	2.275 ± 1.37	0.225 ± 0.18	0.279 ± 0.18	0.769 ± 0.43
Ours dim. (Z=5)	1.694 ± 0.42	3.807 ± 1.14	7.580 ± 1.30	1.807 ± 0.37	4.415 ± 0.30	7.700 ± 0.36
Ours dim. (Z=10)	1.505 ± 0.35	3.529 ± 0.89	6.518 ± 1.63	1.941 ± 0.36	4.320 ± 0.18	7.548 ± 0.28

Table 3: Estimated

\hat{\alpha}

for JOBS and OHIE across baselines and latent dimensions.

Panel of Figure 2 — Figure 2: County-level mean absolute CATE estimates $|\widehat{\tau}_{i}|$ for 2023 across four hazard types. White indicates counties with no event; lighter colors represent higher estimated reporting bias.

Figure 3: Mean absolute CATE values grouped by hazard type. Wildfire events exhibit the highest systematic reporting distortion, followed by tornadic and flooding events.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Scatter plot of the recovered content latents $\hat{Z}$ versus the true latents $Z$ for a representative fold, after aligning dimensions using the closest permutation that maximizes permuted $R^{2}$ .

Figure 5: Sanity Check for the Distribution. Histograms for all key variables:

E_{1}

Z

A

Y_{\text{true}}

Y_{\text{obs}}

, and proxies for a given configuration. Ensures data follows the intended generative model.

Figure 6: Sanity Check - Core Metrics and Correlations. This page summarizes the main structural sanity metrics: $R^{2}(Y_{\text{true}}\mid Z)$ , $R^{2}(Y_{\text{obs}}\mid Y_{\text{true}})$ , the effect of bias latents via $\Delta R^{2}$ , per-dimension $R^{2}$ for each $Z_{i}$ , proxy alignment metrics, as well as the correlation matrix over $\{Z,A,Y\}$ . These diagnostics confirm that the dataset correctly encodes both latent content and latent bias structure.

			$a_{mae}$
			$\alpha=1$		$\alpha=5$		$\alpha=10$
$n$	$d_{z}$	$d_{e}$	Gaussian	Poisson (scaled)	Gaussian	Poisson (scaled)	Gaussian	Poisson (scaled)
500	1	10	1.29±0.33	1.29±0.32	1.39±0.47	1.38±0.49	1.48±0.50	1.46±0.51
500	2	10	1.52±0.22	1.54±0.21	1.63±0.34	1.64±0.34	1.81±0.39	1.85±0.43
500	5	10	1.37±0.27	1.29±0.38	1.09±0.31	1.18±0.28	1.19±0.29	1.13±0.37
500	10	10	1.68±0.34	1.68±0.35	1.72±0.47	1.82±0.40	1.83±0.41	1.83±0.44

Appendix

Table 4: Mean

\pm

Std of

a_{mae}

grouped by

(n,d_{z},d_{e})

rows and

(\alpha,\text{noise})

columns.

			$rmse_{proxies}$
			$\alpha=1$		$\alpha=5$		$\alpha=10$
$n$	$d_{z}$	$d_{e}$	Gaussian	Poisson (scaled)	Gaussian	Poisson (scaled)	Gaussian	Poisson (scaled)
500	1	10	0.20±0.03	0.20±0.03	0.20±0.03	0.20±0.03	0.20±0.03	0.20±0.03
500	2	10	0.21±0.03	0.20±0.02	0.21±0.03	0.20±0.02	0.21±0.03	0.20±0.02
500	5	10	0.19±0.02	0.19±0.02	0.19±0.02	0.19±0.02	0.19±0.02	0.19±0.02
500	10	10	0.15±0.02	0.15±0.02	0.15±0.02	0.15±0.02	0.15±0.02	0.15±0.02

Appendix

Table 5: Mean

\pm

Std of

rmse_{proxies}

grouped by

(n,d_{z},d_{e})

rows and

(\alpha,\text{noise})

columns.

			$z_{mae}$
			$\alpha=1$		$\alpha=5$		$\alpha=10$
$n$	$d_{z}$	$d_{e}$	Gaussian	Poisson (scaled)	Gaussian	Poisson (scaled)	Gaussian	Poisson (scaled)
500	1	10	1.84±0.86	1.82±0.86	1.84±0.86	1.82±0.86	1.84±0.86	1.82±0.86
500	2	10	1.42±0.41	1.50±0.37	1.42±0.41	1.50±0.37	1.42±0.41	1.50±0.37
500	5	10	1.22±0.19	1.23±0.19	1.22±0.19	1.23±0.19	1.22±0.19	1.23±0.19
500	10	10	1.14±0.06	1.14±0.07	1.14±0.06	1.14±0.07	1.14±0.06	1.14±0.07

Appendix

Table 6: Mean

\pm

Std of

z_{mae}

grouped by

(n,d_{z},d_{e})

rows and

(\alpha,\text{noise})

columns.

Variable Type	Name(s)	Description
Outcome	$Y_{\mathrm{obs}}$	Log-transformed emergency department spending ( $0$ – $11.13$ ).
Proxies	$Y_{\mathrm{proxy},1}$ – $Y_{\mathrm{proxy},5}$	Biomedical measurements: systolic BP, diastolic BP, HDL, A1C, PHQ depression score.
Environment $E$ : Treatment	$E_{1}$ (T)	Lottery selection indicator (0/1).
Environment $E$ : State program participation	$E_{2}$ : SNAP household (prenotify) $E_{3}$ : TANF household (prenotify)	SNAP/TANF participation prior to randomization (large variation: 286–2952 unique values).
Environment $E$ : Prior ED utilization	$E_{4}$ : num_visit_pre_cens_ed $E_{5}$ : num_on_pre_cens_ed $E_{6}$ : num_off_pre_cens_ed $E_{7}$ : num_chron_pre_cens_ed $E_{8}$ : num_inj_pre_cens_ed $E_{9}$ : num_hiun_pre_cens_ed $E_{10}$ : num_loun_pre_cens_ed $E_{11}$ : ed_charg_tot_pre_ed $E_{12}$ : charg_tot_pre_ed	Utilization counts and total pre-period ED charges. Reflects burden of chronic, injury-related, heart, head, and psychiatric emergency visits. Many exhibit wide ranges (e.g., $0–$ 180,054 in total charges).
Environment $E$ : In-person clinical measures	$E_{13}$ : tot_med_spend_other_inp	Supplementary in-person medical expenditure measure (0–92,475 across subjects).

Appendix

Table 7: Schema of the OHIE semi-synthetic dataset after preprocessing. Environment variables

E

include treatment, state-program participation, prior emergency-department utilization, and in-person clinical measures. Proxies are clean biomedical measurements and the outcome is log-transformed ED spending.

Variable Type	Name(s)	Description
Outcome	$Y_{\mathrm{obs}}$	Log-transformed 1978 earnings ( $\log(1+\mathrm{re78})$ ), range $0$ – $11.01$ , with 457 unique values.
Proxies	$Y_{\mathrm{proxy},1}$ , $Y_{\mathrm{proxy},2}$	Clean pretreatment earnings: • $Y_{\mathrm{proxy},1}$ (re74): range $0$ – $35{,}040$ , 358 unique values • $Y_{\mathrm{proxy},2}$ (re75): range $0$ – $25{,}142$ , 356 unique values These proxies are unaffected by the injected measurement bias.
Environment $E$ : Treatment	$E_{1}$ (treat)	Random assignment to the job-training intervention (binary 0/1).
Environment $E$ : Demographics	$E_{2}$ : age (16–55; 40 unique values) $E_{3}$ : educ (0–18; 19 unique values) $E_{4}$ : black (0/1) $E_{5}$ : hispan (0/1) $E_{6}$ : married (0/1) $E_{7}$ : nodegree (0/1)	Baseline demographic variables from the NSW/JOBS dataset, representing socioeconomic and educational background.
Bias latent	$A$	Synthetic binary latent generated from a logistic function of the environment variables, used to inject controlled bias into the observed outcome.

Appendix

Table 8: Schema of the JOBS semi-synthetic dataset after preprocessing. Environment variables consist of treatment and demographic covariates; proxies are clean pretreatment earnings; the outcome is log-transformed 1978 earnings.

Category	Variables and description
Identifiers	FIPS (county identifier), Year, Month
Hazard indicators	Hazard type (Flooding, Hurricane/Tropical Storm, Tornado, Wildfire), one-hot encoded as $E_{\text{hazard},k}$
Observed outcome	$Y_{\mathrm{obs}}$ : reported county-level economic loss (property damage, inflation-adjusted to 2023 USD), log-transformed and standardized
Proxy variables	$Y_{\mathrm{proxy},1}$ : fraction of land transitioning from bare ground to water
	$Y_{\mathrm{proxy},2}$ : fraction of built area transitioning to damage-related classes
	$Y_{\mathrm{proxy},3}$ : fraction of built area transitioning to water

Appendix

Table 9: Variables used in the SHELDUS-based real-world case study after preprocessing.

Proximal Causal Inference

See all 9 papers

Estimate Level Adjustment For Inference With Proxies Under Random Distribution Shifts
May 7, 2026Steven Wilkins-Reeves, Alexandra N. M. Darmon, Deeksha SinhaDistribution Shift Robustness Uncertainty Calibration
Central Applied Science, Meta · New York, NY, USA · Menlo Park, CA, USA
proxymate: Diagnosis and Adjustment of Proxy Estimates for Reliable Inference
Jul 27, 2026Alexandra N. M. Darmon, Deeksha Sinha, Steve Wilkins-Reeves +1Causal Inference Uncertainty Calibration
Meta
Proximal Balancing for Causal Effect Estimation under Unmeasured Confounding
Sep 30, 2026Yonghan JungCausal Effect Estimation Proximal Causal Inference
School of Information Sciences University of Illinois Urbana-Champaign

May 7, 2026J/K move · Enter open · S save

Estimate Level Adjustment For Inference With Proxies Under Random Distribution Shifts

Steven Wilkins-Reeves, Alexandra N. M. Darmon, Deeksha Sinha

Central Applied Science, Meta · New York, NY, USA · Menlo Park, CA, USA

Distribution Shift Robustness Uncertainty Calibration

Open paper AskPDF arXiv

In many scientific domains, including experimentation, researchers rely on measurements of proxy outcomes to achieve faster and more frequent reads, especially when the primary outcome of interest is challenging to measure directly. While proxies offer a more readily accessible observation for inference, the ultimate goal is to draw statistical inferences about the primary outcome parameter and proxy data are typically imperfect in some ways. To correct for these imperfections, current statistical inference methods often depend on strict identifying assumptions (such as surrogacy, covariate/label shift, or missingness assumptions). These assumptions can be difficult to validate and may be violated by various additional sources of distribution shift, potentially leading to biased parameter estimates and miscalibrated uncertainty quantification. We introduce an estimate-level framework, inspired by domain adaptation techniques, to empirically calibrate proxy-based inference. This framework models the proxy-primary metric discrepancy as a random effect at the parameter level, estimating its distribution from aggregated historical observations across past domains (e.g., experiments, time periods, or distinct segments). This method avoids the requirement for retaining individual-level response data. Additionally, this adjustment can be layered on top of existing proxy-correction methods (such as prediction-powered inference or importance weighting) to account for additional biases not addressed by those corrections. To manage uncertainty when the number of historical domains is limited, we provide both a method-of-moments estimator and a domain bootstrap procedure. We further validate this approach using publicly available datasets and real-world experiments.

Abstract

Proximal Causal Inference

Explore similar work

Estimate Level Adjustment For Inference With Proxies Under Random Distribution Shifts

proxymate: Diagnosis and Adjustment of Proxy Estimates for Reliable Inference

Proximal Balancing for Causal Effect Estimation under Unmeasured Confounding