Contrastive Linearized Laplace: Loss Geometry, Valid Repairs, and Reliable Cores

Francisco J. Sáez-Maldonado; Luis A. Ortega; Andrés R. Masegosa; Thomas D. Nielsen

cs.LGOct 10, 2026

Contrastive Linearized Laplace: Loss Geometry, Valid Repairs, and Reliable Cores

Authors: Francisco J. Sáez-Maldonado, Luis A. Ortega, Andrés R. Masegosa, Thomas D. Nielsen

Organizations: University of Granada · Aalborg University

Abstract

While contrastive learning is widely used to learn deterministic representations, it does not inherently provide estimates of uncertainty. Post-hoc uncertainty estimation methods, such as the Linearized Laplace approximation (LLA), offer a way to quantify uncertainty without retraining the encoder. However, we show that directly using the LLA induced distribution over the contrastive embedding does not yield reliable pointwise uncertainty estimates for downstream tasks: embedding variance alone does not reflect whether decisions such as nearest-neighbor retrieval or cluster assignment are stable. To address this limitation, we develop a post-hoc framework based on LLA that propagates encoder uncertainty through the downstream task. We characterize the curvature of contrastive objectives and introduce positive-semidefinite repairs where needed to obtain valid local Gaussian approximations without retraining the encoder. We test the proposed approach in metagenomic binning and image classification tasks. For metagenomic binning, posterior coassignment probabilities support consensus partitions and the selection of reliable cores, leading to improvements over existing baselines. Additional image classification experiments show calibration improvements on standard benchmarks and real histopathology datasets.

Figures & tables

Figure 1: Synthetic two-blob example showing that posterior variance is not itself downstream uncertainty. Assignment uncertainty depends on the decision margin and variance along the boundary normal, while score-rank uncertainty depends on the posterior variance of a score difference.

			Output-Hessian repair			Posterior change
Dataset	Loss	Repair	$\lambda_{\min}$	Neg. rate	$\lVert\Delta H\rVert_{F}$	$\Delta$ ACC $\uparrow$	$\Delta$ ECE $\downarrow$	$\Delta$ NLL $\downarrow$
EMNIST	Bernoulli distance	Radial	$-6.22\times 10^{-3}$	$50.2\%$	$22.48$	$+0.009$	$-0.030$	$-0.108$
	Bernoulli distance	Spectral	$-6.22\times 10^{-3}$	$50.2\%$	$3.68\times 10^{-4}$	$+0.008$	$-0.030$	$-0.108$
	Margin	Radial	$-1.22\times 10^{0}$	$13.3\%$	$23.20$	$+0.009$	$-0.030$	$-0.109$
	Margin	Spectral	$-1.22\times 10^{0}$	$13.3\%$	$0.72$	$+0.009$	$-0.030$	$-0.108$
CIFAR-100	Bernoulli distance	Radial	$-3.32\times 10^{-11}$	$49.3\%$	$22.60$	$+0.021$	$-0.080$	$-0.451$

Table 1: Curvature diagnostics and downstream posterior-consensus gains (w.r.t. MAP) on EMNIST and CIFAR100 using a pretrained CLIP encoder, averaged over 5 seeds. Output-Hessian columns report the minimum eigenvalue before repair (

\lambda_{\min}

), the proportion of negative eigenvalues before repair, and the mean per-output-Hessian Frobenius change induced by the repair (

\overline{\|\Delta H\|}_{F}

Figure 2: Genome reliable-core results on Marine and Plant splits. Top panels compare high-quality cluster counts for SemiBin2 and the LLA posterior-consensus variant across F1 thresholds. Bottom panels show high-quality clusters retained as coverage is reduced with different approaches.

		CIFAR-100			EMNIST			SICAP
Encoder	Method	Acc. $\uparrow$	NLL $\downarrow$	ECE $\downarrow$	Acc. $\uparrow$	NLL $\downarrow$	ECE $\downarrow$	Acc. $\uparrow$	NLL $\downarrow$	ECE $\downarrow$
CLIP	MAP	$74.00\pm 0.00$	$1.424\pm 0.000$	$0.165\pm 0.000$	$84.45\pm 0.13$	$0.533\pm 0.010$	$0.064\pm 0.001$	$64.44\pm 0.04$	$1.095\pm 0.000$	$0.175\pm 0.000$
	LLA Sampled	$76.32\pm 0.05$	$0.993\pm 0.004$	$0.084\pm 0.000$	$\mathbf{85.63\pm 0.11}$	$0.454\pm 0.007$	$0.035\pm 0.001$	$66.70\pm 0.45$	$0.930\pm 0.008$	$0.123\pm 0.004$
	Gaussian LLA	$\mathbf{77.34\pm 0.21}$	$\mathbf{0.766\pm 0.002}$	$\mathbf{0.014\pm 0.003}$	$84.95\pm 0.10$	$\mathbf{0.449\pm 0.002}$	$\mathbf{0.020\pm 0.002}$	$\mathbf{66.76\pm 1.51}$	$\mathbf{0.824\pm 0.042}$	$\mathbf{0.062\pm 0.018}$
SimCLR	MAP	$68.25\pm 0.01$	$1.774\pm 0.012$	$0.199\pm 0.001$	$\mathbf{81.79\pm 0.29}$	$0.686\pm 0.017$	$0.079\pm 0.002$	$79.90\pm 0.03$	$0.567\pm 0.001$	$0.093\pm 0.001$
/	LLA Sampled	$\mathbf{69.65\pm 0.12}$	$1.451\pm 0.004$	$0.141\pm 0.001$	$81.74\pm 0.41$	$0.688\pm 0.014$	$0.080\pm 0.004$	$80.44\pm 0.24$	$0.545\pm 0.004$	$0.082\pm 0.003$

Table 2: Classification results on CIFAR-100, EMNIST, and SICAP. Values are mean

\pm

one standard deviation over five seeds. The best result for each dataset and encoder is shown in bold. In the second encoder block, SimCLR is used for CIFAR-100 and EMNIST, and CONCH for SICAP.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Mode	Construction	What it does	What it does not do
Raw GGN	Use $\mathbf{H}_{t}=\nabla^{2}_{f_{t}}\ell_{t}$ directly	Retains the exact output Hessian of the linearized loss	Does not guarantee a PSD precision for distance or margin negatives
Spectral repair	Symmetrize, eigendecompose, and set negative eigenvalues to zero	Projects each output block onto the PSD cone, regardless of the loss	Removes negative curvature; does not turn it into positive evidence
Radial-only repair	Split distance-loss blocks into radial and tangential components; retain radial curvature and damp the tangential component	Gives a structured PSD approximation for distance losses; agrees with spectral repair on active negative margin blocks up to damping	Is not the PSD projection and can remove valid positive tangential curvature from already-PSD positive-pair blocks
Damping/jitter	Add isotropic precision after accumulation	Stabilizes inversion and can make the final matrix strictly positive definite	Does not identify or repair the loss-level curvature pathology
Empirical Fisher	Replace $\mathbf{H}_{t}$ by $\mathbf{g}_{t}\mathbf{g}_{t}^{\top}$	Gives a PSD block by construction	Does not repair the Hessian and can reduce relational curvature to rank one per unit

Appendix

Table 3: Repair and fallback modes considered in this work. The main experiments use repaired GGN curvature, while empirical Fisher and alternative repairs are used as ablations.

Figure 3: Genome reliable-core results on the Marine and Plant splits using EF in place of GGN, with

k=500

retained for k-Means. The top panels compare high-quality cluster counts across F1 thresholds for SemiBin2 and the LLA posterior-consensus variant. The bottom panels show the number of high-quality clusters retained as coverage decreases under the different approaches.

Figure 4: Marginal embedding variance does not determine assignment uncertainty. (a) With identical covariance matrices, a larger margin between the posterior mean and the decision boundary reduces the probability of changing assignment. (b) With identical means and covariance traces, variance perpendicular to the boundary produces greater assignment uncertainty than variance parallel to it. Upper panels show Gaussian embedding posteriors; lower panels show their projections onto the boundary-normal direction. Orange tails indicate assignment changes relative to the posterior mean.

Figure 5: Ranking uncertainty depends on the joint behavior of competing pairwise scores. (a) A query

x_{i}

is compared with candidates

x_{j}

and

x_{k}

, with both scores evaluated under the same posterior encoder draw. (b,c) The scores have identical marginal distributions, but positive covariance stabilizes their difference, whereas negative covariance increases ranking uncertainty. Each row connects the two scores from one illustrative draw; orange rows indicate reversals of the MAP ranking. Reported rank-flip probabilities are computed analytically from the joint Gaussian distribution.

Setting	Frozen encoder	LLA target	Embedding used downstream
CLIP image experiments and ablations	Pretrained CLIP ViT-B/32 visual tower [ Radford et al., 2021 ] , including its learned visual projection. Post-projection normalization is not applied before the LLA target.	CLIP visual projection, a linear map from hidden width 768 to projection dimension 512.	512-dimensional CLIP visual-projection embedding.
SimCLR image experiments	Pretrained ImageNet SimCLRv2 ResNet-50 $1\times$ , SK0 checkpoint [ Chen et al., 2020b ] . Inputs are resized to $224\times 224$ , and the converted ResNet projection output is retained as the LLA target.	Final ResNet $fc$ layer.	Output of the final linear layer.
SemiBin2 genome experiments	Pretrained SemiBin2 global-environment encoder [ Pan et al., 2023 ] , evaluated on 136-dimensional k-mer features.	Last linear layer of the SemiBin2 encoder.	SemiBin2 contig embedding.
SICAPv2/CONCH pathology	Pretrained CONCH-v1.5 vision encoder [ Lu et al., 2024 ] . The LLA implementation exposes the final attention-pooling output map before the contrastive layer normalization.	Final linear output map of the CONCH attention pool.	CONCH patch embedding before the final normalization used by the adapter.

Appendix

Table 4: Frozen encoder architectures and linear layers targeted by LLA. No feature extractor is trained or fine-tuned in any reported experiment.

Method	F1 $>0.9$	F1 $>0.8$	F1 $>0.7$	F1 $>0.6$	F1 $>0.5$
Marine 5
MAP	$20.4\pm 1.1$	$56.8\pm 2.2$	$97.6\pm 3.4$	$150.8\pm 3.4$	$204.8\pm 5.3$
Consensus, EF	$42.0\pm 2.8$	$98.4\pm 2.9$	$158.8\pm 1.9$	$222.2\pm 1.9$	$272.0\pm 0.7$
Consensus, GGN/radial	$42.8\pm 1.5$	$97.0\pm 1.4$	$156.4\pm 1.9$	$217.2\pm 2.7$	$268.2\pm 3.2$
Consensus, GGN/spectral	$42.6\pm 2.1$	$97.8\pm 1.9$	$159.0\pm 0.7$	$220.8\pm 3.7$	$268.2\pm 2.3$
Marine 6

Appendix

Table 5: Numbers of clusters exceeding each aligned F1 threshold for direct SemiBin2 MAP assignments and posterior-consensus solutions, using

K=500

from the cluster-count ablation. Values are mean

\pm

standard deviation over five seeds.

Selector	90%	70%	50%	30%	10%
Marine 5
Consensus + rel. (EF)	$64.4\pm 2.3$	$97.0\pm 3.4$	$118.8\pm 3.1$	$132.2\pm 3.6$	$153.0\pm 5.2$
Consensus + rel. (GGN/radial)	$63.6\pm 2.3$	$96.0\pm 3.1$	$114.6\pm 2.3$	$133.2\pm 4.4$	$152.6\pm 8.6$
Consensus + rel. (GGN/spectral)	$65.0\pm 1.4$	$95.6\pm 3.1$	$118.4\pm 1.9$	$136.0\pm 2.3$	$154.0\pm 3.7$
Consensus + random (EF)	$40.8\pm 2.8$	$41.5\pm 2.4$	$44.3\pm 3.1$	$42.0\pm 2.9$	$50.7\pm 2.0$
Consensus + random (GGN/radial)	$41.1\pm 0.8$	$42.1\pm 1.3$	$44.3\pm 1.4$	$42.5\pm 0.9$	$50.8\pm 1.5$

Appendix

Table 6: Numbers of high-quality clusters (aligned F1

>0.9

) retained at selected coverage levels under consensus and MAP selection rules, using

K=500

from the cluster-count ablation. Values are mean

\pm

standard deviation over five seeds.

Figure 6: Genome reliable-core results on Marine and Plant splits, using

K=100

in k-Means, and using the GGN approximation with spectral repair. Top panels compare high-quality cluster counts for SemiBin2 and the LLA posterior-consensus variant across F1 thresholds. Bottom panels show high-quality clusters retained as coverage is reduced with different approaches.

Figure 7: Genome reliable-core results on Marine and Plant splits, using

K=200

Figure 8: Genome curvature comparison at

K=500

, using GGN with spectral repair as the reference. Points show mean paired differences (alternative minus spectral) across five seeds; error bars indicate one standard deviation of those differences. Macro F1 and ARI differences are expressed on a 0–100 scale, and HQ differences are expressed in cluster counts. Maximum HQ counts are computed separately for each seed over the 10–100% coverage grid using consensus-based reliability selection before taking paired differences.

Figure 9: Image classification posterior-sample ablation. We report the results as the difference between the performance of our proposal and that of the MAP solution. No repair step is required in these experiments, as the InfoNCE loss is used.

Figure 10: SemiBin2 posterior-sample ablation. We report the results as the difference between the performance of our proposal and that of the MAP solution.

Figure 11: SemiBin2 clustering-

K

ablation. Values are mean

\pm

one standard deviation over five seeds at full coverage. MAP denotes standard SemiBin2 clustering and is shown once per

K

using the EF/none run outputs.

Contrastive Learning

Contrastive learning trains representations by bringing related examples together and separating unrelated ones for recognition, retrieval, and transfer.

See all 322 papers

Between the Layers Lies the Truth: Uncertainty Estimation in LLMs Using Intra-Layer Local Information Scores
Mar 17, 2026Zvi N. Badash, Yonatan Belinkov, Moti FreimanUncertainty Quantification LLM Uncertainty Estimation
Faculty of Data and Decision Sciences, Technion — Israel Institute of Technology, Haifa, Israel · Faculty of Computer Science, Technion — Israel Institute of Technology, Haifa, Israel · Faculty of Biomedical Engineering, Technion — Israel Institute of Technology, Haifa, Israel
From Entropy to Calibrated Uncertainty: Training Language Models to Reason About Uncertainty
Mar 6, 2026Azza Jenane, Nassim Walha, Lukas Kuhn +1LLM Uncertainty Estimation Language Model Calibration
German Cancer Research Center (DKFZ) German Cancer Consortium (DKTK) Goethe University Frankfurt, Germany
The Loss Is Not Enough: Sampling Conditions and Inductive Bias in Contrastive Representation Learning
Jun 2, 2026Justinas Zaliaduonis, Patrick Putzky, Till Richter +1Contrastive Learning Self-Supervised Learning
Technical University of Munich, Munich, Germany · Stanford University, Stanford, USA · Merantix Momentum GmbH, Berlin, Germany +2

Mar 17, 2026J/K move · Enter open · S save

Between the Layers Lies the Truth: Uncertainty Estimation in LLMs Using Intra-Layer Local Information Scores

Zvi N. Badash, Yonatan Belinkov, Moti Freiman

Faculty of Data and Decision Sciences, Technion — Israel Institute of Technology, Haifa, Israel · Faculty of Computer Science, Technion — Israel Institute of Technology, Haifa, Israel · Faculty of Biomedical Engineering, Technion — Israel Institute of Technology, Haifa, Israel

Uncertainty Quantification LLM Uncertainty Estimation

Open paper AskPDF arXiv

Large language models (LLMs) are often confidently wrong, making reliable uncertainty estimation (UE) essential. Output-based heuristics are cheap but brittle, while probing internal representations is effective yet high-dimensional and hard to transfer. We propose a compact, per-instance UE method that scores cross-layer agreement patterns in internal representations using a single forward pass. Across three models, our method matches probing in-distribution, with mean diagonal differences of at most

-1.8

AUPRC percentage points and

+4.9

Brier score points. Under cross-dataset transfer, it consistently outperforms probing, achieving off-diagonal gains up to

+2.86

AUPRC and

+21.02

Brier points. Under 4-bit weight-only quantization, it remains robust, improving over probing by

+1.94

AUPRC points and

+5.33

Brier points on average. Beyond performance, examining specific layer--layer interactions reveals differences in how disparate models encode uncertainty. Altogether, our UE method offers a lightweight, compact means to capture transferable uncertainty in LLMs.

Abstract

Contrastive Learning

Explore similar work

Between the Layers Lies the Truth: Uncertainty Estimation in LLMs Using Intra-Layer Local Information Scores

From Entropy to Calibrated Uncertainty: Training Language Models to Reason About Uncertainty

The Loss Is Not Enough: Sampling Conditions and Inductive Bias in Contrastive Representation Learning