Can Metadata Fix the Gauge? Calibration Turns Sparse Multi-Source Learning from Sparse PCA into Sparse Mean Recovery

Yirong Xiang; Yibo Zhou

cs.LGOct 19, 2026

Can Metadata Fix the Gauge? Calibration Turns Sparse Multi-Source Learning from Sparse PCA into Sparse Mean Recovery

Authors: Yirong Xiang, Yibo Zhou

Organizations: Air Force Medical Center · Beihang University

Abstract

Source labels tell us where a sample came from, but not how that source oriented its coordinates. In a sparse Gaussian multi-source model with source-specific sign gauges, that distinction changes the learning problem. Without calibration, pooling across sources erases the mean signal, and the source-labelled experiment reduces to sparse PCA. With task-independent relative-gauge metadata, synchronization aligns sources and the same observations become a sparse mean problem. We formalize the value of metadata through AlignedMass, which quantifies how much first-order signal a metadata-induced alignment restores. Source labels create only limited AlignedMass, whereas valid calibration can create AlignedMass proportional to the full sample size. This yields a statistical separation in micro-source regimes and, under standard sparse-PCA hardness assumptions, a computational separation as well. Synthetic experiments check the predicted scaling; controlled real-feature and physical sensor studies examine sign cancellation, separately acquired calibration, and reuse across tasks.

Figures & tables

Panel of Figure 1 — Figure 1: Mechanism tests. Calibration follows sparse-mean scaling; uninformative or mismatched metadata fails to restore alignment. The calibrated/source-label gap is largest for weak sources. Gray dashed lines mark unit strength.

$N_{\mathrm{src}}$	$A_{q}/\sqrt{M_{q}}$	Corr	Succ90
1	$0.825\pm 0.068$	$0.632\pm 0.038$	$0.460\pm 0.050$
4	$0.828\pm 0.031$	$0.832\pm 0.024$	$0.630\pm 0.049$
16	$0.783\pm 0.015$	$0.880\pm 0.011$	$0.600\pm 0.049$
32	$0.800\pm 0.011$	$0.917\pm 0.008$	$0.800\pm 0.040$
128	$0.790\pm 0.005$	$0.916\pm 0.006$	$0.770\pm 0.042$

Table 1: At fixed aligned mass , increasing aligned source count stabilizes amplitude and improves recovery. Entries are mean

\pm

SE over

100

seeds.

Method	Corr	F1
Oracle calibration	$0.928{\pm}0.002$	$0.928{\pm}0.002$
Same sync reused	$0.928{\pm}0.002$	$0.928{\pm}0.002$
No calibration	$0.018{\pm}0.001$	$0.019{\pm}0.001$
Random metadata reused	$0.020{\pm}0.002$	$0.021{\pm}0.002$
Shuffled metadata reused	$0.022{\pm}0.001$	$0.023{\pm}0.001$
Source diagonal PCA	$0.040{\pm}0.002$	$0.049{\pm}0.002$

Table 2: One calibration graph reused across

32

independent sparse tasks. Entries are mean

\pm

task-level SE conditional on the graph.

Method	Digits Acc.	Digits $M_{q}/n$	Breast Acc.	Breast $M_{q}/n$
Clean no gauge	$0.968\pm 0.001$	$1.000\pm 0.000$	$0.933\pm 0.003$	$1.000\pm 0.000$
Oracle calibration	$0.968\pm 0.001$	$1.000\pm 0.000$	$0.933\pm 0.003$	$1.000\pm 0.000$
Relative sync calibration	$0.968\pm 0.001$	$1.000\pm 0.000$	$0.932\pm 0.003$	$0.997\pm 0.003$
Noisy abs. calibration, $10\%$ flips	$0.874\pm 0.006$	$0.650\pm 0.018$	$0.851\pm 0.011$	$0.667\pm 0.031$
No calibration	$0.500\pm 0.004$	$0.003\pm 0.000$	$0.501\pm 0.005$	$(1.180\pm 0.234)\times 10^{-5}$
Random metadata	$0.525\pm 0.005$	$0.011\pm 0.002$	$0.556\pm 0.008$	$0.022\pm 0.005$

Table 3: Controlled real-feature experiments. Features and labels are real; source gauges and calibration metadata are imposed. Entries are mean

\pm

SE over

30

seeds.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

$(d,k)$	$\rho_{\rm cal}$	$n$	Oracle Corr	Oracle Succ90	Sync Corr	Sync Succ90
$(1000,20)$	0.250	242	$0.095\pm 0.010$	0.000	$0.098\pm 0.013$	0.000
	0.500	484	$0.192\pm 0.012$	0.000	$0.220\pm 0.018$	0.000
	1.000	966	$0.377\pm 0.021$	0.000	$0.378\pm 0.016$	0.000
	2.000	1930	$0.680\pm 0.016$	0.033	$0.668\pm 0.014$	0.000
	4.000	3858	$0.913\pm 0.008$	0.833	$0.900\pm 0.008$	0.700
	8.000	7716	$0.997\pm 0.002$	1.000	$0.995\pm 0.003$	1.000

Appendix

Table 4: Source-Gaussian calibrated phase transition. Corr entries are mean

\pm

SE, and Succ90 is reported in adjacent columns. Each entry uses

30

seeds.

$s$	$s\mu^{2}$	$L$	$\mu^{4}ns/K_{\mathrm{sp}}$	Oracle	Sync	Diag+PCA	Diag F1	Sync Succ90
1	0.020	19649	0.080	$0.955\pm 0.004$	$0.962\pm 0.007$	$0.031\pm 0.009$	$0.038\pm 0.009$	1.000
2	0.040	9824	0.160	$0.960\pm 0.006$	$0.960\pm 0.007$	$0.022\pm 0.006$	$0.033\pm 0.007$	0.967
4	0.080	4912	0.320	$0.962\pm 0.006$	$0.963\pm 0.006$	$0.032\pm 0.007$	$0.050\pm 0.009$	1.000
8	0.160	2456	0.640	$0.958\pm 0.008$	$0.950\pm 0.005$	$0.045\pm 0.010$	$0.065\pm 0.011$	1.000
16	0.320	1228	1.280	$0.950\pm 0.007$	$0.953\pm 0.007$	$0.084\pm 0.012$	$0.110\pm 0.012$	0.967
32	0.640	614	2.560	$0.957\pm 0.008$	$0.952\pm 0.007$	$0.194\pm 0.024$	$0.165\pm 0.013$	0.967

Appendix

Table 5: Source-Gaussian micro-source separation. Entries are mean

\pm

SE, except Sync Succ90, which reports the success rate. Each entry uses

30

seeds.

Method	Balanced test accuracy	$M_{q}/n$
Oracle mounting polarity	$0.921\pm 0.004$	$1.000\pm 0.000$
Relative calibration + synchronization	$0.902\pm 0.006$	$0.916\pm 0.015$
No calibration	$0.558\pm 0.011$	$0.044\pm 0.010$
Shuffled metadata	$0.553\pm 0.012$	$0.040\pm 0.009$

Appendix

Table 6: Physical calibration reused across four tasks. Entries are estimates

\pm

source-cluster bootstrap SE. Mounting signs are unavailable to the relative-calibration procedure.

Sparse Recovery

Sparse recovery reconstructs signals or models with few active components from limited, noisy, or indirect measurements.

See all 76 papers

Rethinking Calibration for High-Sparsity LLM Pruning: From Single Sources to Capability Coverage
Jun 2, 2026Hu Xu, Zhaolong Xing, Congcong Liu +5LLM Pruning Model Calibration
Shanghai Jiao Tong University
CALIBUDGET: Calibration-Guided Source Allocation for Fixed-Budget Mixed-Reasoning Adaptation
Sep 29, 2026Yupeng Chang, Yuan WuData Mixture Optimization Adaptive Sampling
School of Artificial Intelligence, Jilin University · Key Laboratory of Symbolic Computation and Knowledge Engineering, Jilin University
Measurement-Gated Provenance Attenuation for Frozen EEG Representations
Sep 26, 2026Anuar Aimoldin, Yankai Chen, Ayana Mussabayeva +2Invariant Representation Learning Distribution Shift
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)

Jun 2, 2026J/K move · Enter open · S save

Rethinking Calibration for High-Sparsity LLM Pruning: From Single Sources to Capability Coverage

Hu Xu, Zhaolong Xing, Congcong Liu, Jiaxing Wang+4

Shanghai Jiao Tong University

LLM Pruning Model Calibration

Open paper AskPDF arXiv

Post-training LLM pruning relies on a small calibration set, usually chosen heuristically and judged by WikiText perplexity or by mean accuracy on general-domain commonsense benchmarks. This single-axis view hides capability-specific effects: at 60% sparsity, strategies separated by

2.85

points in averaged commonsense accuracy diverge by

51.9

points in code retention, and across 15 sources no single one preserves all capabilities. We treat calibration as a capability-coverage problem. Balanced, a uniform four-way mixture under a fixed budget, tests the principle on real corpora: it outperforms every evaluated single source, and its margin over C4 grows from

+4.6

points at 30% sparsity to

+18.8

at 60%. The advantage holds across model scale and a second model family, and transfers to held-out benchmarks disjoint from calibration construction. When aligned corpora are unavailable, IGSP approximates that mixture from the base model and the evaluation taxonomy alone, outperforming prior self-generation methods by

7.23

12.79

points on the LLaMA main suite. On Qwen2.5-7B, IGSP exceeds Balanced on both the main suite and held-out benchmarks. Capability coverage is a first-order design variable for high-sparsity pruning.

Abstract

Sparse Recovery

Explore similar work

Rethinking Calibration for High-Sparsity LLM Pruning: From Single Sources to Capability Coverage

CALIBUDGET: Calibration-Guided Source Allocation for Fixed-Budget Mixed-Reasoning Adaptation

Measurement-Gated Provenance Attenuation for Frozen EEG Representations