Margin and Consistency Supervision for Calibrated and Robust Vision Models

Salim Khazem

cs.CVMar 6, 2026

Margin and Consistency Supervision for Calibrated and Robust Vision Models

Authors: Salim Khazem

Organizations: Talan Research Center, Paris, France.

Abstract

Deep vision classifiers often achieve high accuracy while remaining poorly calibrated and fragile under small distribution shifts. We present Margin and Consistency Supervision (MaCS), a simple, architecture-agnostic regularization framework that jointly enforces logit-space separation and local prediction stability. MaCS augments cross-entropy with (i) a hinge-squared margin penalty that enforces a target logit gap between the correct class and the strongest competitor, and (ii) a consistency regularizer that minimizes the KL divergence between predictions on clean inputs and mildly perturbed views. We provide a unifying theoretical analysis showing that increasing classification margin while reducing local sensitivity formalized via a Lipschitz-type stability proxy yields improved generalization guarantees and a provable robustness radius bound scaling with the margin-to-sensitivity ratio. Across several image classification benchmarks and several backbones spanning CNNs and Vision Transformers, MaCS consistently improves calibration (lower ECE and NLL) and robustness to common corruptions while preserving or improving top-1 accuracy. Our approach requires no additional data, no architectural changes, and negligible inference overhead, making it an effective drop-in replacement for standard training objectives.

Figures & tables

Figure 1: Per-dataset model curves across methods. Each line corresponds to a model and traces accuracy across training objectives, highlighting method-specific gains.

Figure 2: Overview of MaCS training. The model processes both clean input $x$ and perturbed input $\tilde{x}=T(x)$ . The total loss combines cross-entropy, a margin penalty encouraging $\gamma(x)\geq\Delta$ , and a KL-based consistency term enforcing prediction stability.

Method	Accuracy $\uparrow$	Robust $\uparrow$
CIFAR-10
Baseline (CE)	87.63 ±0.51	39.92 ±0.33
Focal Loss	87.35 ±0.67	40.94 ±0.95
Label Smoothing	88.85 ±0.54	41.87 ±0.53
Mixup	89.63 ±0.60	42.68 ±0.36
MaCS (Ours)	91.10 ±0.55	43.48 ±0.28

Table 1: Comparison on CIFAR-10/100 with ResNet-50. MaCS achieves the best accuracy and robustness. Results: mean

{}_{\pm\text{std}}

over 3 seeds.

Figure 3: Accuracy improvement of MaCS over baseline (cross-entropy) across all dataset–model configurations. Each line connects the baseline (left) to MaCS (right) accuracy. MaCS improves over baseline in the large majority of settings, with the largest gains on CIFAR and Food-101.

Perturbation	Acc $\uparrow$	ECE $\downarrow$	Robust $\uparrow$
None (CE only)	63.41	24.57	20.00
Noise only	67.82	4.21	23.14
Blur only	68.15	3.89	23.52
Noise + Blur	69.23	3.13	24.60

Table 2: Perturbation ablation on CIFAR-100 (ResNet-50). Combining noise and blur yields the best robustness.

Dataset	Method	ECE $\downarrow$	NLL $\downarrow$
CIFAR-10	Baseline (CE)	9.10 ±0.43	0.779 ±0.213
	Focal Loss	3.31 ±0.24	0.408 ±0.017
	Label Smoothing	5.44 ±0.26	0.463 ±0.013
	Mixup	3.26 ±0.32	0.379 ±0.012
	MaCS (Ours)	2.48 ±0.22	0.329 ±0.019
CIFAR-100	Baseline (CE)	24.57 ±0.82	2.458 ±0.614

Table 3: Calibration metrics on CIFAR-10/100 (ResNet-50). Lower is better. MaCS achieves the best ECE and NLL.

	ECE (%) $\downarrow$		NLL $\downarrow$
Method	Pre-TS	Post-TS	Pre-TS	Post-TS
Baseline (CE)	24.57	3.42	2.458	1.385
Focal Loss	12.86	3.18	1.550	1.342
Label Smooth.	3.14	2.89	1.576	1.356
Mixup	7.52	2.95	1.407	1.298
AugMix	5.83	2.76	1.382	1.275

Table 4: Calibration before and after temperature scaling (TS) on CIFAR-100 (ResNet-50). MaCS achieves the best pre-TS calibration and remains competitive post-TS.

Figure 4: Negative log-likelihood comparison for ResNet-50 on CIFAR-10/100. MaCS improves NLL relative to baseline but does not always outperform the strongest calibration baselines.

Dataset	Model	Baseline	AugMix	MaCS	MaCS+Aug
C10-C	ConvNeXt-T	46.1 ±.8	50.8 ±.6	51.2 ±.9	53.4 ±.7
	EffNet-B0	36.3 ±.7	38.1 ±.5	37.4 ±.6	39.2 ±.4
	MobNetV3	33.4 ±.2	35.9 ±.3	36.8 ±.2	38.2 ±.2
	ResNet-34	40.5 ±.5	43.2 ±.4	42.7 ±.4	46.4 ±.3
	ResNet-50	42.4 ±.1	44.1 ±.3	46.1 ±.0	48.4 ±.2
C100-C	ConvNeXt-T	24.8 ±.5	32.0 ±.5	33.3 ±.7	35.1 ±.4

Table 5: Corruption robustness (%) on CIFAR-10-C and CIFAR-100-C. Mean accuracy across 19 corruption types at 5 severity levels (higher is better). Best in bold .

Figure 5: Corruption robustness on CIFAR-10-C and CIFAR-100-C (ResNet-50). MaCS consistently outperforms all baselines including Mixup.

Method	$\\|\text{logit}\\|_{2}$	Max Logit	Margin $\gamma$
Baseline (CE)	12.4	8.7	2.31
Focal Loss	9.8	6.2	1.89
Label Smooth.	10.1	6.8	2.15
Mixup	11.2	7.4	2.52
MaCS	13.1	9.2	3.64

Table 6: Logit statistics on CIFAR-100 test set (ResNet-50). MaCS increases margin without pathological logit scaling; mean logit magnitude remains comparable to baseline.

Method	Noise †	Blur †	Weather	Digital	Avg
Baseline	31.2	38.5	47.8	52.1	42.4
AugMix	35.8	41.2	49.3	54.2	44.1
MaCS	33.9	40.8	48.6	53.4	43.1
MaCS+Aug	37.2	43.1	50.8	55.6	45.4
Excluding noise/blur (no overlap):
Baseline	–	–	47.8	52.1	49.9

Table 7: CIFAR-10-C robustness (%) by corruption family (ResNet-50). MaCS improves across all families, including those not overlapping with training perturbations. † Noise/blur overlap with training perturbations.

Training Data	Baseline	MaCS	$\Delta$
10%	73.1	75.2	+2.1
25%	81.1	82.2	+1.1
50%	85.3	85.7	+0.4
100%	87.6	91.1	+3.5

Table 8: Data efficiency comparison on CIFAR-10 (ResNet-50). MaCS improves accuracy across all data fractions.

Model	Focal	LS	Mixup	MaCS	AugMix
ResNet-34	1.05 $\times$	1.00 $\times$	1.05 $\times$	2.11 $\times$	3.03 $\times$
ResNet-50	1.00 $\times$	1.05 $\times$	1.01 $\times$	2.04 $\times$	3.45 $\times$
ConvNeXt-T	1.00 $\times$	1.03 $\times$	1.04 $\times$	1.74 $\times$	2.39 $\times$
MobileNetV3	1.09 $\times$	1.00 $\times$	1.07 $\times$	1.96 $\times$	2.74 $\times$
ViT-S/16	1.01 $\times$	1.02 $\times$	1.08 $\times$	2.07 $\times$	3.46 $\times$
Swin-T	1.03 $\times$	1.00 $\times$	1.04 $\times$	1.85 $\times$	2.76 $\times$

Table 9: Training overhead relative to baseline cross-entropy. MaCS incurs

\sim

\times

overhead from one extra forward pass; AugMix requires

\sim

\times

. Inference cost is identical for all methods.

Figure 6: Component ablation visualization on a representative ResNet-50 setting. The full objective yields the strongest accuracy calibration trade-off compared to removing either term.

Method	Margin $\gamma$ $\uparrow$	Sensitivity $\hat{L}$ $\downarrow$	Ratio $\uparrow$
Baseline	2.31	4.87	0.47
MaCS	3.64	3.52	1.03

Table 10: Empirical margin-to-sensitivity ratio on CIFAR-100 test set (ConvNeXt-Tiny). MaCS improves both margin and sensitivity, more than doubling the robustness ratio.

Configuration	Acc	ECE	NLL
Baseline (CE only)	75.53	19.29	2.112
+ Margin only ( $\lambda_{c}=0$ )	75.89	16.45	1.856
+ Consistency only ( $\lambda_{m}=0$ )	76.21	14.32	1.623
Full MaCS	77.65	12.62	1.399

Table 11: Ablation study on CIFAR-100 (ConvNeXt-Tiny). Both margin and consistency losses contribute to improvements. The full objective achieves best calibration.

Distribution Shift Robustness

Distribution shift robustness studies how to keep machine-learning predictions reliable when deployment data differ from training data.

See all 280 papers

Too Sharp, Too Sure: When Calibration Follows Curvature
Apr 22, 2026Alessandro Morosini, Matea Gjika, Tomaso Poggio +1Loss Landscape Sharpness Margin-Based Learning
Massachusetts Institute of Technology, Cambridge, MA, USA.
Certified Training for Convolutional Perturbations
Jul 20, 2026Benedikt Brückner, Alessio LomuscioImage Corruption Robustness Neural Network Robustness
Safe Intelligence · Imperial College London
Towards Calibrating Prompt Tuning of Vision-Language Models
Feb 22, 2026Ashshak Sharifdeen, Fahad Shamshad, Muhammad Akhtar Munir +6VLM Calibration Prompt Learning for VLMs
Mohamed bin Zayed University of AI · University of Colombo · Michigan State University

Apr 22, 2026J/K move · Enter open · S save

Too Sharp, Too Sure: When Calibration Follows Curvature

Alessandro Morosini, Matea Gjika, Tomaso Poggio, Pierfrancesco Beneventano

Massachusetts Institute of Technology, Cambridge, MA, USA.

Loss Landscape Sharpness Margin-Based Learning Model Calibration Neural Network Training Dynamics Statistical Learning Theory

Open paper AskPDF arXiv

Modern neural networks can achieve high accuracy while remaining poorly calibrated, producing confidence estimates that do not match empirical correctness. Yet calibration is often treated as a post-hoc attribute. We take a different perspective: we study calibration as a training-time phenomenon on small vision tasks, and ask whether calibrated solutions can be obtained reliably by intervening on the training procedure. We identify a tight coupling between calibration, curvature, and margins during training of deep networks under multiple gradient-based methods. Empirically, Expected Calibration Error (ECE) closely tracks curvature-based sharpness throughout optimization. Mathematically, we show that both ECE and Gauss--Newton curvature are controlled, up to problem-specific constants, by the same margin-dependent exponential tail functional along the trajectory. Guided by this mechanism, we introduce a margin-aware training objective that explicitly targets robust-margin tails and local smoothness, yielding improved out-of-sample calibration across optimizers without sacrificing accuracy.

Abstract

Distribution Shift Robustness

Explore similar work

Too Sharp, Too Sure: When Calibration Follows Curvature

Certified Training for Convolutional Perturbations

Towards Calibrating Prompt Tuning of Vision-Language Models