Margin and Consistency Supervision for Calibrated and Robust Vision Models

Salim Khazem

cs.CVMar 6, 2026

Margin and Consistency Supervision for Calibrated and Robust Vision Models

Authors: Salim Khazem

Organizations: Talan Research Center, Paris, France.

Abstract

Deep vision classifiers often achieve high accuracy while remaining poorly calibrated and fragile under small distribution shifts. We present Margin and Consistency Supervision (MaCS), a simple, architecture-agnostic regularization framework that jointly enforces logit-space separation and local prediction stability. MaCS augments cross-entropy with (i) a hinge-squared margin penalty that enforces a target logit gap between the correct class and the strongest competitor, and (ii) a consistency regularizer that minimizes the KL divergence between predictions on clean inputs and mildly perturbed views. We provide a unifying theoretical analysis showing that increasing classification margin while reducing local sensitivity formalized via a Lipschitz-type stability proxy yields improved generalization guarantees and a provable robustness radius bound scaling with the margin-to-sensitivity ratio. Across several image classification benchmarks and several backbones spanning CNNs and Vision Transformers, MaCS consistently improves calibration (lower ECE and NLL) and robustness to common corruptions while preserving or improving top-1 accuracy. Our approach requires no additional data, no architectural changes, and negligible inference overhead, making it an effective drop-in replacement for standard training objectives.

Figures & tables

Figure 1: Per-dataset model curves across methods. Each line corresponds to a model and traces accuracy across training objectives, highlighting method-specific gains.

Figure 2: Overview of MaCS training. The model processes both clean input $x$ and perturbed input $\tilde{x}=T(x)$ . The total loss combines cross-entropy, a margin penalty encouraging $\gamma(x)\geq\Delta$ , and a KL-based consistency term enforcing prediction stability.

Method	Accuracy $\uparrow$	Robust $\uparrow$
CIFAR-10
Baseline (CE)	87.63 ±0.51	39.92 ±0.33
Focal Loss	87.35 ±0.67	40.94 ±0.95
Label Smoothing	88.85 ±0.54	41.87 ±0.53
Mixup	89.63 ±0.60	42.68 ±0.36
MaCS (Ours)	91.10 ±0.55	43.48 ±0.28

Table 1: Comparison on CIFAR-10/100 with ResNet-50. MaCS achieves the best accuracy and robustness. Results: mean

{}_{\pm\text{std}}

over 3 seeds.

Figure 3: Accuracy improvement of MaCS over baseline (cross-entropy) across all dataset–model configurations. Each line connects the baseline (left) to MaCS (right) accuracy. MaCS improves over baseline in the large majority of settings, with the largest gains on CIFAR and Food-101.

Perturbation	Acc $\uparrow$	ECE $\downarrow$	Robust $\uparrow$
None (CE only)	63.41	24.57	20.00
Noise only	67.82	4.21	23.14
Blur only	68.15	3.89	23.52
Noise + Blur	69.23	3.13	24.60

Table 2: Perturbation ablation on CIFAR-100 (ResNet-50). Combining noise and blur yields the best robustness.

Dataset	Method	ECE $\downarrow$	NLL $\downarrow$
CIFAR-10	Baseline (CE)	9.10 ±0.43	0.779 ±0.213
	Focal Loss	3.31 ±0.24	0.408 ±0.017
	Label Smoothing	5.44 ±0.26	0.463 ±0.013
	Mixup	3.26 ±0.32	0.379 ±0.012
	MaCS (Ours)	2.48 ±0.22	0.329 ±0.019
CIFAR-100	Baseline (CE)	24.57 ±0.82	2.458 ±0.614

Table 3: Calibration metrics on CIFAR-10/100 (ResNet-50). Lower is better. MaCS achieves the best ECE and NLL.

	ECE (%) $\downarrow$		NLL $\downarrow$
Method	Pre-TS	Post-TS	Pre-TS	Post-TS
Baseline (CE)	24.57	3.42	2.458	1.385
Focal Loss	12.86	3.18	1.550	1.342
Label Smooth.	3.14	2.89	1.576	1.356
Mixup	7.52	2.95	1.407	1.298
AugMix	5.83	2.76	1.382	1.275

Table 4: Calibration before and after temperature scaling (TS) on CIFAR-100 (ResNet-50). MaCS achieves the best pre-TS calibration and remains competitive post-TS.

Figure 4: Negative log-likelihood comparison for ResNet-50 on CIFAR-10/100. MaCS improves NLL relative to baseline but does not always outperform the strongest calibration baselines.

Dataset	Model	Baseline	AugMix	MaCS	MaCS+Aug
C10-C	ConvNeXt-T	46.1 ±.8	50.8 ±.6	51.2 ±.9	53.4 ±.7
	EffNet-B0	36.3 ±.7	38.1 ±.5	37.4 ±.6	39.2 ±.4
	MobNetV3	33.4 ±.2	35.9 ±.3	36.8 ±.2	38.2 ±.2
	ResNet-34	40.5 ±.5	43.2 ±.4	42.7 ±.4	46.4 ±.3
	ResNet-50	42.4 ±.1	44.1 ±.3	46.1 ±.0	48.4 ±.2
C100-C	ConvNeXt-T	24.8 ±.5	32.0 ±.5	33.3 ±.7	35.1 ±.4

Table 5: Corruption robustness (%) on CIFAR-10-C and CIFAR-100-C. Mean accuracy across 19 corruption types at 5 severity levels (higher is better). Best in bold .

Figure 5: Corruption robustness on CIFAR-10-C and CIFAR-100-C (ResNet-50). MaCS consistently outperforms all baselines including Mixup.

Method	$\\|\text{logit}\\|_{2}$	Max Logit	Margin $\gamma$
Baseline (CE)	12.4	8.7	2.31
Focal Loss	9.8	6.2	1.89
Label Smooth.	10.1	6.8	2.15
Mixup	11.2	7.4	2.52
MaCS	13.1	9.2	3.64

Table 6: Logit statistics on CIFAR-100 test set (ResNet-50). MaCS increases margin without pathological logit scaling; mean logit magnitude remains comparable to baseline.

Method	Noise †	Blur †	Weather	Digital	Avg
Baseline	31.2	38.5	47.8	52.1	42.4
AugMix	35.8	41.2	49.3	54.2	44.1
MaCS	33.9	40.8	48.6	53.4	43.1
MaCS+Aug	37.2	43.1	50.8	55.6	45.4
Excluding noise/blur (no overlap):
Baseline	–	–	47.8	52.1	49.9

Table 7: CIFAR-10-C robustness (%) by corruption family (ResNet-50). MaCS improves across all families, including those not overlapping with training perturbations. † Noise/blur overlap with training perturbations.

Training Data	Baseline	MaCS	$\Delta$
10%	73.1	75.2	+2.1
25%	81.1	82.2	+1.1
50%	85.3	85.7	+0.4
100%	87.6	91.1	+3.5

Table 8: Data efficiency comparison on CIFAR-10 (ResNet-50). MaCS improves accuracy across all data fractions.

Model	Focal	LS	Mixup	MaCS	AugMix
ResNet-34	1.05 $\times$	1.00 $\times$	1.05 $\times$	2.11 $\times$	3.03 $\times$
ResNet-50	1.00 $\times$	1.05 $\times$	1.01 $\times$	2.04 $\times$	3.45 $\times$
ConvNeXt-T	1.00 $\times$	1.03 $\times$	1.04 $\times$	1.74 $\times$	2.39 $\times$
MobileNetV3	1.09 $\times$	1.00 $\times$	1.07 $\times$	1.96 $\times$	2.74 $\times$
ViT-S/16	1.01 $\times$	1.02 $\times$	1.08 $\times$	2.07 $\times$	3.46 $\times$
Swin-T	1.03 $\times$	1.00 $\times$	1.04 $\times$	1.85 $\times$	2.76 $\times$

Table 9: Training overhead relative to baseline cross-entropy. MaCS incurs

\sim

\times

overhead from one extra forward pass; AugMix requires

\sim

\times

. Inference cost is identical for all methods.

Figure 6: Component ablation visualization on a representative ResNet-50 setting. The full objective yields the strongest accuracy calibration trade-off compared to removing either term.

Method	Margin $\gamma$ $\uparrow$	Sensitivity $\hat{L}$ $\downarrow$	Ratio $\uparrow$
Baseline	2.31	4.87	0.47
MaCS	3.64	3.52	1.03

Table 10: Empirical margin-to-sensitivity ratio on CIFAR-100 test set (ConvNeXt-Tiny). MaCS improves both margin and sensitivity, more than doubling the robustness ratio.

Configuration	Acc	ECE	NLL
Baseline (CE only)	75.53	19.29	2.112
+ Margin only ( $\lambda_{c}=0$ )	75.89	16.45	1.856
+ Consistency only ( $\lambda_{m}=0$ )	76.21	14.32	1.623
Full MaCS	77.65	12.62	1.399

Table 11: Ablation study on CIFAR-100 (ConvNeXt-Tiny). Both margin and consistency losses contribute to improvements. The full objective achieves best calibration.

Distribution Shift Robustness

Distribution shift robustness studies how to keep machine-learning predictions reliable when deployment data differ from training data.

See all 280 papers

Apr 22, 2026

Too Sharp, Too Sure: When Calibration Follows Curvature

Modern neural networks can achieve high accuracy while remaining poorly calibrated, producing confidence estimates that do not match empirical correctness. Yet calibration is often treated as a post-hoc attribute. We take a different perspective: we study calibration as a training-time phenomenon on small vision tasks, and ask whether calibrated solutions can be obtained reliably by intervening on the training procedure. We identify a tight coupling between calibration, curvature, and margins during training of deep networks under multiple gradient-based methods. Empirically, Expected Calibration Error (ECE) closely tracks curvature-based sharpness throughout optimization. Mathematically, we show that both ECE and Gauss--Newton curvature are controlled, up to problem-specific constants, by the same margin-dependent exponential tail functional along the trajectory. Guided by this mechanism, we introduce a margin-aware training objective that explicitly targets robust-margin tails and local smoothness, yielding improved out-of-sample calibration across optimizers without sacrificing accuracy.

Alessandro Morosini, Matea Gjika, Tomaso Poggio +1

Massachusetts Institute of Technology, Cambridge, MA, USA.

Loss Landscape Sharpness Margin-Based Learning Model Calibration Neural Network Training Dynamics Statistical Learning Theory

Ask PDF

Jul 20, 2026

Certified Training for Convolutional Perturbations

Vision models have been found to be susceptible to perturbations such as motion blur induced at runtime by a shaking camera. This impedes their deployment in critical applications since phenomena such as slightly blurred vision might lead to failures, for example an object detector missing objects. While methods such as data augmentation or Adversarial Training can improve empirical robustness, they lack formal safety guarantees, making it difficult to identify and mitigate hidden vulnerabilities. We introduce a novel Certified Training approach that leverages an efficient encoding of convolutional perturbations to train provably robust models. Our method significantly outperforms Adversarial Training, achieving, for example, over 80% robust accuracy against motion blur of reasonable intensity on CIFAR10 while maintaining comparable standard accuracy.

Benedikt Brückner, Alessio Lomuscio

Safe Intelligence · Imperial College London

Image Corruption Robustness Neural Network Robustness Certified Robustness

Ask PDF

Feb 22, 2026

Towards Calibrating Prompt Tuning of Vision-Language Models

Prompt tuning of large-scale vision-language models such as CLIP enables efficient task adaptation without updating model weights. However, it often leads to poor confidence calibration and unreliable predictive uncertainty. We address this problem by proposing a calibration framework that enhances predictive reliability while preserving the geometry of the pretrained CLIP embedding space, which is required for robust generalization. Our approach extends the standard cross-entropy loss with two complementary regularizers: (1) a mean-variance margin penalty that stabilizes inter-class logit margins by maximizing their average while minimizing dispersion, mitigating underconfidence and overconfidence spikes; and (2) a text moment-matching loss that aligns the first and second moments of tuned text embeddings with their frozen CLIP counterparts, preserving semantic dispersion crucial for generalization. Through extensive experiments across 7 prompt-tuning methods and 11 diverse datasets, we demonstrate that our approach significantly reduces the Expected Calibration Error (ECE) compared to competitive calibration techniques on both base and novel classes

Ashshak Sharifdeen, Fahad Shamshad, Muhammad Akhtar Munir +6

Mohamed bin Zayed University of AI · University of Colombo · Michigan State University

VLM Calibration Prompt Learning for VLMs Confidence Estimation for VLMs

Ask PDF

Abstract

Distribution Shift Robustness

Explore similar work

Too Sharp, Too Sure: When Calibration Follows Curvature

Certified Training for Convolutional Perturbations

Towards Calibrating Prompt Tuning of Vision-Language Models