CÆNL: Collapse-Based Active Neural Learning with Objective-Aligned Representation Control

Saad Alqithami

cs.LGOct 13, 2026

CÆNL: Collapse-Based Active Neural Learning with Objective-Aligned Representation Control

Authors: Saad Alqithami

Abstract

Active learning changes the representations used to acquire subsequent labels. We study this interaction through Collapse-Based Active Neural Learning (CÆNL), which separates acquisition, Dynamic Collapse Regularization (DCR), and its coefficient policy. DCR penalizes a normalized within-class to between-class scatter residual above a frozen reference, with a separate class-separation safeguard. Fixed, scheduled, deterministic-feedback, and learned-feedback policies apply this objective. The analysis establishes scatter invariances, conditional local descent, and coefficient bounds, and shows why clean geometry alone cannot guarantee input robustness. Six-seed classification comparisons are complemented by a 104-run study with separate control selection and ten-seed evaluations on each of two backbones. At 60% annotations on ImageNet-100, entropy acquisition with DCR and MACC-Lite reaches 73.420 +/- 0.512% accuracy on ResNet-50 and 73.086 +/- 0.397% on ResNet-18 (mean +/- seed SD), exceeding entropy alone by 1.742 and 1.834 percentage points. Both gains survive Holm correction under paired-t and exact sign-flip tests, with unchanged feedback settings across backbones. Independently tuned constants attain slightly higher mean accuracy; the paired comparisons establish neither adaptive superiority nor equivalence. Separate six-seed spectral studies yield a small fixed-regularization C4 loss reduction, worse CIFAR-10 diffusion FID, and inconclusive AudioCaps captioning effects. Earlier classification studies find near-zero AutoAttack accuracy at L-infinity radius 8/255 and no added benefit from learned control. The results support the evaluated combined classification treatment and identify the task dependence of geometry-informed regularization.

Figures & tables

Method	Principal strength or focus	Distinction or limitation for this study	Matched top-1 (%)
Center loss / DeepLDA [ 14 , 13 ]	Direct discriminative representation objectives	No matched coefficient-control experiment here	—
Supervised contrastive learning [ 15 ]	Label-based attraction and separation of embeddings	Different objective; not a tested alternative loss here	—
VICReg [ 16 ]	Preserves variance while reducing redundancy	Self-supervised objective; distinct from class-conditional contraction	—
GradNorm / uncertainty weighting [ 5 , 22 ]	Adaptive balancing of multiple task losses	Different feedback signals and optimization goals; not executed here	—
BADGE [ 3 ]	Uncertainty and gradient diversity	Acquisition criterion; does not modulate the scatter objective	$71.110\pm 0.467$
Noise Stability [ 18 ]	Perturbation-based acquisition signal	Higher measured acquisition cost in this implementation	$71.023\pm 0.249$

Table 1: Positioning relative to representative acquisition, representation, and loss-weighting methods. Numerical entries are ImageNet-100 clean accuracy at 60% annotations in the earlier six-seed studies (mean

\pm

seed SD), not numbers pooled from different published benchmarks. A dash denotes a conceptual comparison that was not evaluated here.

Panel of Figure 1 — Figure 1: The classification workflow of CÆNL. (a) Acquisition expands the annotated set between training phases. (b) The regularizer and adaptive controllers use the same normalized scatter statistic, while the separation safeguard penalizes insufficient between-class separation. Fixed strength, deterministic feedback, and learned feedback are alternatives. The initial fixed strength is 0.03; the later control study selects 0.10 and also tests preset schedules (Section 6.4 ). Full MACC credits the terminal reward and keeps its acquisition-policy head disabled. (c) The fixed holdout supplies Full-MACC feedback without task-model gradient updates; official evaluation is separate. Solid arrows indicate execution or applied control; dashed arrows indicate measured signals. Language, diffusion, and audio use their task-specific spectral formulations.

Condition	Acquisition	Representation control
Random	Uniform random selection	None
Entropy	Highest predictive entropy	None
BADGE	Exact factorized weight-gradient distances	None; independent implementation, bias excluded
Noise Stability	Perturbed-output deviations with greedy coverage	None; five noise draws, independent implementation
CÆNL acquisition only	Predicted-class diagonal Mahalanobis score	None
Random + aligned fixed DCR	Uniform random selection	Aligned objective, $\lambda_{l}=0.03$

Table 2: Initial ImageNet-100 comparison. All nine conditions use seeds 801–806 and the same 10–60% annotation schedule. “None” disables both aligned penalties; all conditions record geometry. The three follow-up conditions are reported separately.

Task	Loss and observed statistic	Fixed/adaptive coefficients	Update convention
ImageNet-100 classification	Aligned normalized within/between log ratio; fixed separation safeguard	Fixed $0.03$ in the initial study and selected $0.10$ in the later study; adaptive initialization $0.03$ ; bounds $[0,0.10]$	Aligned Lite uses the leaky rule; Full uses the aligned reward; windows of at most 100 steps and phase-boundary handling
GPT-2/C4	Spectral loss with $k=128$ , ranks 16 and 4 at h6/h12; separate covariance-spectrum feedback	Fixed $0.01$ ; adaptive initialization $0.005$ ; bounds $[0,0.08]$	Spectral control every 25 steps; Lite step size $0.002$ and deadband $0.1$
CIFAR-10 diffusion	Spectral loss with $k=64$ , rank-16 target; full EMA-covariance monitor	Fixed $0.01$ ; adaptive initialization $0.01$ ; bounds $[0,0.05]$	Spectral control every 50 steps; Lite step size $0.001$ and deadband $0.1$
AudioCaps captioning	Spectral loss with $k=64$ , rank-16 targets at prefix/encoder; different sample axes and zero padding	Fixed $0.01$ ; adaptive initialization $0.01$ ; bounds $[0,0.05]$	Spectral control every 50 steps and final step; terminal reward credited without an unused next action

Table 3: Task-specific regularization and feedback conventions. “Lite” and “Full” denote different task-specific implementations, not an unchanged controller transferred across all tasks. Task protocols provide the remaining settings.

Control	Holdout top-1 (%)	Holdout cross-entropy	Selected
Constant 0.003	$75.408\pm 2.519$	$0.976\pm 0.066$	—
Constant 0.010	$75.619\pm 1.003$	$0.955\pm 0.050$	—
Constant 0.030	$75.197\pm 0.790$	$0.963\pm 0.056$	—
Constant 0.100	$76.567\pm 1.121$	$0.886\pm 0.058$	Yes
Decreasing, peak 0.03	$74.829\pm 0.638$	$0.969\pm 0.047$	—
Decreasing, peak 0.10	$76.461\pm 0.632$	$0.948\pm 0.045$	Yes

Table 4: All candidate controls on the three ResNet-50 tuning seeds 41001–41003. Values are final training-holdout mean

\pm

sample SD; each seed has 633 holdout images. Selection maximizes mean accuracy within each family, with cross-entropy and identifier tie-breaks. These are selection scores, not independent performance estimates. Official validation is not evaluated during tuning.

Backbone	Treatment	Clean top-1 (%)	Cross-entropy	ECE
ResNet-50	Entropy	$71.678\pm 0.808$	$1.137\pm 0.024$	$0.079\pm 0.007$
ResNet-50	Entropy + MACC-Lite	$73.420\pm 0.512$	$1.020\pm 0.014$	$0.058\pm 0.006$
ResNet-50	Entropy + tuned fixed	$73.608\pm 0.474$	$1.010\pm 0.019$	$0.054\pm 0.006$
ResNet-50	Entropy + tuned schedule	$73.022\pm 0.371$	$1.052\pm 0.019$	$0.064\pm 0.004$
ResNet-18	Entropy	$71.252\pm 0.397$	$1.197\pm 0.022$	$0.095\pm 0.004$
ResNet-18	Entropy + MACC-Lite	$73.086\pm 0.397$	$1.091\pm 0.018$	$0.082\pm 0.003$

Table 5: Control-study outcomes at 60% annotations on ImageNet-100. Values are mean

\pm

sample SD over ten paired seeds per backbone. Every treatment uses entropy acquisition. Fixed and scheduled strengths are selected using separate ResNet-50 tuning seeds; all settings are frozen for confirmation and transfer.

Backbone	Contrast	Difference	95% CI	Adjusted $t$ $p$	Adjusted exact $p$
ResNet-50	Lite $-$ entropy	$+1.742$	$[+1.354,+2.130]$	$1.57\!\times\!10^{-5}$	0.01172
ResNet-50	Lite $-$ tuned fixed	$-0.188$	$[-0.529,+0.153]$	0.2436	0.25781
ResNet-50	Lite $-$ tuned schedule	$+0.398$	$[+0.073,+0.723]$	0.065	0.08203
ResNet-18	Lite $-$ entropy	$+1.834$	$[+1.558,+2.110]$	$6.74\!\times\!10^{-7}$	0.01172
ResNet-18	Lite $-$ tuned fixed	$-0.334$	$[-0.741,+0.073]$	0.1929	0.20312
ResNet-18	Lite $-$ tuned schedule	$+0.324$	$[+0.155,+0.493]$	0.007541	0.03125

Table 6: All six predeclared final-top-1 contrasts in the control study. Effects and individual, unadjusted 95% paired-

t

intervals are in percentage points. Each test column is separately Holm-adjusted across the full six-comparison family.

Figure 2: Fresh-seed control comparison. Panels (a) and (b) show mean official validation accuracy over ten seeds at measured acquisition budgets. Panel (c) shows all six final Lite-minus-control effects and individual 95% paired-

t

intervals. The zero line denotes no difference; interval exclusion does not by itself imply multiplicity-adjusted significance. The test results are given in Table 6 .

Figure 3: Applied Lite strengths from native training traces. Each curve is the ten-seed mean coefficient in successive windows of at most 100 steps within each phase; separate line styles identify layer3 and layer4. Vertical separators mark acquisition-round boundaries, and the horizontal reference is the tuned constant strength 0.10. Coefficients carry state across phases while responding to their respective moving geometry residuals.

Condition	Study	Clean top-1 (%)	AutoAttack $8/255$ (%)	Cross-entropy	ECE
Random	I	$69.293\pm 0.334$	$0.013\pm 0.010$	$1.272712\pm 0.021145$	$0.094945\pm 0.002917$
Entropy	I	$71.463\pm 0.320$	$0.010\pm 0.011$	$1.150591\pm 0.020293$	$0.079698\pm 0.003849$
BADGE	I	$71.110\pm 0.467$	$0.003\pm 0.008$	$1.149581\pm 0.007022$	$0.077088\pm 0.004058$
Noise Stability	I	$71.023\pm 0.249$	$0.003\pm 0.008$	$1.162821\pm 0.006167$	$0.078703\pm 0.002375$
CÆNL acquisition only	I	$70.190\pm 0.240$	$0.013\pm 0.010$	$1.210545\pm 0.010404$	$0.085552\pm 0.004501$
Random + aligned fixed DCR	I	$70.113\pm 0.452$	$0.017\pm 0.008$	$1.217939\pm 0.021315$	$0.085964\pm 0.006228$

Table 7: The 12 earlier six-seed ImageNet-100 conditions at 60% annotations. Values are mean

\pm

sample SD over six paired seeds; clean and standard AutoAttack accuracy use all 5,000 validation images. Study I denotes the initial nine-condition comparison, A the later acquisition-rule ablation, B the later UHerding comparison, and C the later entropy-plus-fixed control. The table is descriptive: study-specific inferential families remain separate. Bold identifies the highest observed mean clean accuracy, not a universal superiority claim.

Figure 4: Acquisition and aligned regularization under the common ImageNet-100 protocol. Panels (a) and (b) show measured accuracy trajectories on identical axes; panel (c) enlarges the final 60% endpoint for the original nine conditions and the UHerding-Margin follow-up. Markers and bars represent six-seed means and sample SDs, not confidence intervals. All trajectories start from the same 10% checkpoint. Fixed, Lite, and Full denote aligned DCR conditions; entropy regularization follow-ups appear in Figure 5 . Connecting lines guide the eye, and no new area-under-curve test or inferential family is introduced. Geometry and UH-Margin abbreviate geometry acquisition and UHerding-Margin.

Target minus reference	Difference (pp)	95% CI (pp)	Adjusted $t$ $p$	Adjusted exact $p$
Aligned fixed DCR $-$ acquisition only	+0.850	$[+0.430,+1.270]$	0.0208	0.2188
Aligned MACC-Lite $-$ aligned fixed DCR	+1.080	$[+0.447,+1.713]$	0.0309	0.2188
Aligned Full MACC $-$ aligned fixed DCR	$-0.237$	$[-0.714,+0.241]$	0.2589	0.2813
Aligned MACC-Lite $-$ entropy	+0.657	$[+0.015,+1.299]$	0.0931	0.2188
Aligned Full MACC $-$ entropy	$-0.660$	$[-0.869,-0.451]$	0.0032	0.2188
Acquisition only $-$ random	+0.897	$[+0.389,+1.405]$	0.0309	0.2188

Table 8: All seven original clean-accuracy contrasts in percentage points. Intervals are individual unadjusted 95% paired-

t

intervals. Each

p

-value column is separately Holm-adjusted across seven contrasts. The exact tests have the resolution limit described in Section 6.7 .

Reference condition	Gain (pp)	95% CI (pp)	Adjusted $t$ $p$	Adjusted exact $p$
Entropy without aligned regularization	+1.897	$[+1.625,+2.169]$	0.0000198	0.0625
Geometry acquisition + aligned MACC-Lite	+1.240	$[+0.707,+1.773]$	0.001875	0.0625

Table 9: Two acquisition-rule clean-accuracy contrasts. Positive values favor entropy with aligned DCR and Lite. Intervals are individual unadjusted paired-

t

intervals; each test family is Holm-adjusted across these two contrasts. This family was specified after inspection of the initial comparison.

Figure 5: Matched entropy treatments across annotation budgets and seeds. (a) Mean clean accuracy and sample SD across six seeds. (b) Mean and sample SD of within-seed differences from entropy; the common 10% initialization has zero gain and is omitted. (c) Individual final-budget accuracies, with lines joining the same seed across conditions; adjacent filled markers and bars show mean

\pm

sample SD. Overlapping seed values are retained. These bars are not confidence intervals. Fixed DCR uses

\lambda_{l}=0.03

and the same separation penalty as Lite. Lite is not uniformly better at early budgets, but exceeds both comparators in all six final-budget seeds. Only the final 60% comparisons enter the declared follow-up tests. All points derive from recorded measurements.

Target minus reference	Difference	95% CI	Raw $t$ $p$	Adjusted $t$ $p$	Exact raw / adjusted
Entropy + fixed $-$ entropy	+0.800	$[+0.498,+1.102]$	0.001047	0.001442	0.03125 / 0.0625
Entropy + Lite $-$ entropy + fixed	+1.097	$[+0.714,+1.479]$	0.000721	0.001442	0.03125 / 0.0625

Table 10: The six-seed fixed-control follow-up’s two clean-accuracy contrasts. Differences and individual unadjusted 95% paired-

t

intervals are in percentage points. Holm correction is applied separately to the two paired-

t

and the two exact tests in this family. Prior families are unchanged.

Target minus reference	Difference	95% CI	Adjusted $t$ $p$	Adjusted exact $p$
UHerding-Margin $-$ entropy	+0.283	$[-0.430,+0.997]$	0.3540	0.3125
Entropy + aligned Lite $-$ UHerding-Margin	+1.613	$[+0.957,+2.270]$	0.002923	0.0625

Table 11: The UHerding comparison’s two clean-accuracy contrasts, in percentage points. Intervals are individual unadjusted 95% paired-

t

intervals; each test column is Holm-adjusted across these two comparisons only.

Condition	Mean log trace ratio
Random	$1.3881\pm 0.0197$
Entropy	$1.3916\pm 0.0170$
Acquisition only	$1.3864\pm 0.0151$
Random + aligned fixed	$1.0607\pm 0.0218$
CÆNL + aligned fixed	$0.9767\pm 0.0200$
CÆNL + aligned Lite	$0.4937\pm 0.0390$

Table 12: Final all-class normalized geometry, mean

\pm

seed SD. The statistic averages layer3 and layer4

\log((W+\epsilon)/(B+\epsilon))

Condition	NLL (nats/token)	Perplexity	Pooled token ECE	Token accuracy
Baseline	$3.384212\pm 0.000239$	$29.49473\pm 0.00705$	$0.012324\pm 0.000426$	$0.380067\pm 0.000123$
Fixed spectral DCR	$3.383919\pm 0.000253$	$29.48611\pm 0.00746$	$0.012216\pm 0.000438$	$0.380029\pm 0.000083$
Spectral MACC-Lite	$3.386159\pm 0.000220$	$29.55222\pm 0.00651$	$0.013131\pm 0.000396$	$0.379920\pm 0.000098$
Spectral Full MACC	$3.384448\pm 0.000462$	$29.50170\pm 0.01362$	$0.012438\pm 0.000584$	$0.380040\pm 0.000111$

Table 13: Unused-local-shard C4 evaluation of the same 24 final GPT-2 checkpoints. Values are mean

\pm

sample SD over six training seeds, with 1,999,872 next-token predictions per checkpoint. The formulations are spectral, not aligned classification DCR.

Condition minus baseline	Difference	95% CI	Raw $t$ $p$	Holm $t$ $p$	Raw exact $p$	Holm exact $p$
Fixed spectral DCR	$-2.92505$	$[-3.28129,-2.56881]$	$4.4239\times 10^{-6}$	$8.8477\times 10^{-6}$	0.03125	0.09375
Spectral MACC-Lite	$+19.47289$	$[+18.78979,+20.15599]$	$8.9651\times 10^{-9}$	$2.6895\times 10^{-8}$	0.03125	0.09375
Spectral Full MACC	$+2.36236$	$[-1.87778,+6.60250]$	0.21152	0.21152	0.21875	0.21875

Table 14: All three unused-shard C4 NLL contrasts. Differences and individual 95% paired-

t

intervals are in

10^{-4}

nats/token; negative differences favor regularization. Holm adjustment is within the three NLL contrasts, separately for each test. Both test families use the same three contrasts; intervals are unadjusted.

Condition	FID $\downarrow$	KID $\times 10^{3}\downarrow$	Inception score $\uparrow$
Pretrained reference	$10.512\pm 0.100$	$9.153\pm 0.124$	$8.479\pm 0.039$
Unregularized fine-tuning	$\mathbf{8.057\pm 0.048}$	$6.605\pm 0.084$	$8.692\pm 0.043$
Fixed spectral DCR	$8.213\pm 0.070$	$6.726\pm 0.105$	$8.724\pm 0.049$
Spectral MACC-Lite	$8.472\pm 0.100$	$6.939\pm 0.115$	$8.728\pm 0.055$
Spectral Full MACC	$8.211\pm 0.116$	$6.744\pm 0.088$	$8.729\pm 0.050$

Table 15: Diffusion confirmation with 50,000 generated images per condition and seed. Values are mean

\pm

sample SD over six seed streams. FID is primary. Reference variation is sampling variation for one unchanged pretrained model.

Target minus reference	Difference	95% interval	Adjusted $t$ $p$	Adjusted exact $p$
Baseline $-$ pretrained reference	$-2.45468$	$[-2.58346,-2.32589]$	$2.6769\times 10^{-7}$	0.125
Fixed DCR $-$ baseline	+0.15601	$[+0.11357,+0.19845]$	0.0004481	0.125
MACC-Lite $-$ baseline	+0.41472	$[+0.31931,+0.51012]$	0.0003004	0.125
Full MACC $-$ baseline	+0.15431	$[+0.06205,+0.24658]$	0.0077204	0.125

Table 16: All four diffusion FID contrasts. Positive differences indicate worse FID for the target. Intervals are individual unadjusted paired-

t

intervals. Each test family is Holm-adjusted over these four comparisons.

Condition	CIDEr-D	BLEU-4	METEOR	SPICE	SPIDEr
Baseline	$0.6736\pm 0.0220$	$0.2401\pm 0.0137$	$0.2328\pm 0.0043$	$0.1664\pm 0.0046$	$0.4200\pm 0.0132$
Fixed spectral DCR	$0.6745\pm 0.0185$	$0.2384\pm 0.0114$	$0.2335\pm 0.0022$	$0.1650\pm 0.0041$	$0.4198\pm 0.0110$
Spectral MACC-Lite	$0.6668\pm 0.0212$	$0.2378\pm 0.0108$	$0.2320\pm 0.0035$	$0.1664\pm 0.0020$	$0.4166\pm 0.0112$
Spectral Full MACC	$0.6703\pm 0.0175$	$0.2382\pm 0.0094$	$0.2319\pm 0.0023$	$0.1659\pm 0.0041$	$0.4181\pm 0.0100$

Table 17: AudioCaps results on the fixed available subset: mean

\pm

sample SD over seeds 1301–1306. Each run uses 866 test clips with five references each. Metrics retain their native scales; CIDEr-D is primary.

Target minus baseline	Difference	95% interval	Adjusted $t$ $p$	Adjusted exact $p$
Fixed spectral DCR	+0.000896	$[-0.016040,+0.017833]$	1.0000	1.0000
Spectral MACC-Lite	$-0.006781$	$[-0.028950,+0.015389]$	1.0000	1.0000
Spectral Full MACC	$-0.003329$	$[-0.015313,+0.008656]$	1.0000	1.0000

Table 18: All three primary audio CIDEr-D contrasts. Positive differences favor the target. Intervals are individual unadjusted paired-

t

intervals; each test family is Holm-adjusted across three comparisons.

Method	Training compute	Feedback inference	Acquisition	Final AutoAttack
Random	5093.24	182.25	0.85	792.52
Entropy	5092.81	182.45	490.83	800.95
BADGE	5093.97	182.55	523.26	775.80
Noise Stability	5093.61	182.52	2942.89	777.36
Acquisition only	5093.23	182.38	739.69	798.50
Random + aligned fixed	5240.51	182.30	0.86	811.51

Table 19: Recorded ImageNet-100 timing components in seconds per conceptual condition run, averaged over six seeds. Training and feedback include their allocated shared-initialization costs. These are component, not complete application, timings.

Condition	Five-phase duration (s)	Recorded phase peak (bytes)
Entropy	5,255.02	6,066,439,680
Entropy + aligned MACC-Lite	5,391.31	6,066,701,824
Difference	+136.29	+262,144

Table 20: Post-initialization training time and peak allocation. Duration is the mean sum over five post-initialization training phases. Peaks describe PyTorch allocation in those phases, not isolated controller memory.

Question	Supported finding	Limit of the inference
Objective–feedback alignment	Shared supervised scatter residual, local isolated descent, and bounded coefficients	Necessity of alignment, global optimization, and closed-loop stability are not established
Active learning	Matched acquisition comparisons; entropy can be combined with aligned regularization	No universal acquisition ranking or equal-performance 40% annotation saving
MACC-Lite	Final-budget gains over entropy on two backbones; earlier gains over specified 0.03 controls	No established superiority to the tuned constant, equivalence, or isolated causal benefit of adaptation
Full MACC	Policy updates execute, without an established added classification benefit	Results concern the evaluated controller; learned query thresholds are not tested
Adversarial robustness	Near-zero full-set AutoAttack accuracy at the specified radius	No meaningful robustness gain or certification follows from the geometric objectives
Language modeling	Small fixed spectral-DCR loss reduction on an unused local C4 shard	Effect is specific to this spectral objective and language-modeling protocol

Table 21: Interpretation and scope of the reported evidence. Analytical statements retain their stated assumptions; empirical findings apply to the evaluated protocols.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Seed	Entropy	Entropy + fixed	Entropy + Lite	Fixed $-$ entropy	Lite $-$ fixed	Lite robust correct
801	71.14	72.38	73.10	+1.24	+0.72	0/5,000
802	71.76	72.22	73.48	+0.46	+1.26	1/5,000
803	71.32	72.00	73.30	+0.68	+1.30	0/5,000
804	71.74	72.78	73.70	+1.04	+0.92	0/5,000
805	71.08	71.70	73.34	+0.62	+1.64	0/5,000
806	71.74	72.50	73.24	+0.76	+0.74	0/5,000

Appendix

Table 22: Paired entropy-acquisition treatments at 60% annotations on ImageNet-100. Clean accuracies are percentages and differences are percentage points; fixed aligned DCR uses coefficient 0.03. The entropy and Lite reference runs are shared with the corresponding follow-up comparisons. Robust-correct counts use standard AutoAttack at

L_{\infty}

radius

8/255

on all 5,000 validation images. Fixed DCR has 0/5,000 robust-correct images in every seed.

Seed	Entropy	UHerding-Margin	Entropy + aligned Lite	UHerding robust correct
801	71.14	71.58	73.10	0/5,000
802	71.76	72.64	73.48	1/5,000
803	71.32	72.38	73.30	0/5,000
804	71.74	71.46	73.70	0/5,000
805	71.08	71.40	73.34	1/5,000
806	71.74	71.02	73.24	0/5,000

Appendix

Table 23: Paired UHerding-Margin comparison at 60% annotations on ImageNet-100. Clean accuracies are percentages; the summary row reports mean

\pm

sample SD. Entropy and entropy-plus-Lite are the same reference runs as in Table 22 . UHerding-Margin uses the fixed shared-initial ResNet-50 features specified in Section 6.6 . Robust-correct counts concern UHerding-Margin under standard AutoAttack at

L_{\infty}

radius

8/255

on all 5,000 validation images.

Target minus reference	Difference (pp)	95% CI (pp)	Adjusted $t$ $p$	Adjusted exact $p$
Aligned fixed DCR $-$ acquisition only	$+0.0033$	$[-0.0125,+0.0191]$	1.0000	1.0000
Aligned MACC-Lite $-$ aligned fixed DCR	$-0.0100$	$[-0.0215,+0.0015]$	0.5291	1.0000
Aligned Full MACC $-$ aligned fixed DCR	$-0.0100$	$[-0.0215,+0.0015]$	0.5291	1.0000
Aligned MACC-Lite $-$ entropy	$-0.0033$	$[-0.0119,+0.0052]$	1.0000	1.0000
Aligned Full MACC $-$ entropy	$-0.0033$	$[-0.0191,+0.0125]$	1.0000	1.0000
Acquisition only $-$ random	$0.0000$	$[-0.0133,+0.0133]$	1.0000	1.0000

Appendix

Table 24: All seven original AutoAttack accuracy contrasts at

L_{\infty}

radius

8/255

on 5,000 ImageNet-100 validation images. Effects and individual, unadjusted 95% paired-

t

intervals are in percentage points over six paired seeds. Each test column is separately Holm-adjusted across these seven comparisons. Aligned conditions use geometric acquisition unless random acquisition is named explicitly.

Seed	Baseline	Fixed	Lite	Full
1301	0.699902	0.706662	0.695297	0.697474
1302	0.683473	0.656485	0.683658	0.675072
1303	0.634166	0.656615	0.658886	0.647493
1304	0.681798	0.678712	0.663435	0.671009
1305	0.672890	0.676179	0.633839	0.655220
1306	0.669446	0.672400	0.665875	0.675435

Appendix

Table 25: Paired AudioCaps CIDEr-D outcomes on the 866 retained test recordings, on the native metric scale. Fixed, Lite, and Full denote the task-specific spectral conditions; larger values indicate better caption agreement. Each row corresponds to one shared training seed.

Backbone	Seed	Entropy	Lite	Tuned fixed	Tuned schedule
ResNet-50	42001	70.78	73.16	73.10	72.40
ResNet-50	42002	71.22	72.68	73.32	72.64
ResNet-50	42003	72.04	73.56	73.30	72.98
ResNet-50	42004	72.76	73.66	74.68	73.40
ResNet-50	42005	70.28	72.84	73.52	72.62
ResNet-50	42006	71.50	73.62	73.58	72.92

Appendix

Table 26: Control-study final top-1 accuracy (%) at 60% annotations on all 5,000 ImageNet-100 validation images. Each row pairs four entropy-acquisition treatments under one seed. Fixed and scheduled strengths were selected before these runs and transferred unchanged to ResNet-18.

Backbone	Contrast	Difference	95% CI	Adjusted $t$ $p$	Adjusted exact $p$
ResNet-50	Lite $-$ entropy	$+1.742$	$[+1.358,+2.125]$	$1.42\!\times\!10^{-5}$	0.01172
ResNet-50	Lite $-$ fixed	$-0.188$	$[-0.527,+0.151]$	0.2406	0.25391
ResNet-50	Lite $-$ schedule	$+0.398$	$[+0.075,+0.722]$	0.06367	0.08789
ResNet-18	Lite $-$ entropy	$+1.836$	$[+1.564,+2.108]$	$5.86\!\times\!10^{-7}$	0.01172
ResNet-18	Lite $-$ fixed	$-0.328$	$[-0.732,+0.075]$	0.1978	0.20703
ResNet-18	Lite $-$ schedule	$+0.322$	$[+0.155,+0.490]$	0.00744	0.03125

Appendix

Table 27: Content-duplicate sensitivity on 4,995 validation images. Effects and individual, unadjusted 95% paired-

t

intervals are in percentage points. Fixed and schedule denote the selected controls. Each test column is separately Holm-adjusted across the same six contrasts as in the primary analysis; all adjusted decisions are unchanged. Positive effects favor Lite.

Condition minus baseline	Difference	95% CI	Raw $t$ $p$	Holm $t$ $p$	Raw exact $p$	Holm exact $p$
Perplexity: differences in $10^{-2}$
Fixed spectral DCR	$-0.86260$	$[-0.96757,-0.75763]$	$4.4061\times 10^{-6}$	$8.8122\times 10^{-6}$	0.03125	0.09375
Spectral MACC-Lite	$+5.74906$	$[+5.54778,+5.95034]$	$8.8781\times 10^{-9}$	$2.6634\times 10^{-8}$	0.03125	0.09375
Spectral Full MACC	$+0.69705$	$[-0.55390,+1.94800]$	0.21147	0.21147	0.21875	0.21875
Pooled token ECE: differences in $10^{-4}$
Fixed spectral DCR	$-1.07333$	$[-4.14827,+2.00161]$	0.41069	0.78043	0.37500	0.68750

Appendix

Table 28: Secondary C4 comparisons on the unused local validation shard. Each panel reports three regularized-condition-minus-baseline contrasts with individual, unadjusted 95% paired-

t

intervals. The paired-

t

and exact sign-flip

p

-values are separately Holm-adjusted within each panel. Multiply effects and interval limits by the scale shown in the panel heading to recover their original units; token ECE and accuracy use proportions rather than percentages.

Active Learning

Active learning selects the examples, labels, features, or experiments most useful for improving a model under limited budgets.

See all 284 papers

May 19, 2026

Neural Collapse by Design: Learning Class Prototypes on the Hypersphere

Supervised classification has a theoretical optimum, Neural Collapse (NC), yet neither of its two dominant paradigms reaches it in practice. Cross entropy (CE) leaves radial degrees of freedom unconstrained and converges to a degenerate geometry, while supervised contrastive learning (SCL) drives features toward NC during pretraining but discards this structure in a post hoc linear probing phase. We show that both paradigms are different appearances of the same method that contrasts prototypes on the unit hypersphere, and that closing the gap requires fixing each at its point of failure. From the CE side, we propose NTCE and NONL, two normalized losses that import contrastive optimization's missing ingredients into classifier learning: a large effective negative set and decoupled alignment and uniformity terms. From the SCL side, we prove that SCL's objective already optimizes throughout training for a principled classifier whose weights are the class mean embeddings, making linear probing both redundant and harmful. Empirically, on four benchmarks including ImageNet-1K, NTCE and NONL surpass CE accuracy, closely approximate NC (

\geq 95\%

), and match CE's converged NC on 4/5 metrics in under

7.5\%

of its iterations, while SCL with fixed prototypes matches linear probing without the hours-long classifier training phase. The learned geometry yields

+5.5\%

mean relative improvement in transfer learning, up to

+8.7\%

under severe class imbalance, and improved robustness to corruptions on ImageNet-C. Our work recasts supervised learning as prototype learning on the hypersphere, with NC reached by design.

Panagiotis Koromilas, Theodoros Giannakopoulos, Mihalis A. Nicolaou +1

University of Athens · NSCR Demokritos · University of Cyprus +2

Prototype-Based Classification Hyperspherical Representation Learning Neural Collapse Supervised Contrastive Learning

Ask PDF

May 21, 2026

Human-Centered Learning Mechanics: Observable Entropy Control under Protected-Risk Constraints

Entropy reduction is not a safety criterion: it can leave predictions unchanged or accompany increasing risk for a protected group. We study entropy-regulated learning under separately specified task, risk, and intervention constraints. A noise-calibrated log-determinant functional yields an exact spectral force formula and a metric decomposition into observable and prediction-null velocities. For scalar feedback, we characterize the complete interval of coefficients preserving a task-descent fraction and meeting an entropy-rate request; restricting intervention to observable directions can make that interval empty. For several protected risks, we derive an exact minimax certificate of feasibility within a specified subspace and norm budget, an exact robust extension for uncertain constraint rows, and a cone-projection formula for the entropy reduction remaining after protection. These formulas use standard convex geometry but identify when apparently active entropy control has no admissible effect. Barrier comparisons, finite-step curvature and gradient-error margins, and fresh-data audits give distinct conditional guarantees for risk thresholds. Human authority enters through the specification and release policy; the mathematics does not identify compression with alignment or certify human well-being. A memorization counterexample separates fresh-input information from sample-level information, and a noisy sample-code assumption supports a conditional generalization bound. Reproducible mathematical checks and a small real-data experiment expose both feasible interventions and failures. The resulting theory characterizes admissible entropy regulation and its limitations, rather than asserting universal accuracy or safety improvements.

Kim Phuc Tran

Univ. Lille, ENSAIT, ULR 2461 – GEMTEX – Génie et Matériaux Textiles, F-59000 Lille, France · International Chair in DS & XAI, International Research Institute for Artificial Intelligence and Data Science, Dong A University, Danang, Vietnam

Representation Learning Neural Network Training Dynamics Entropy Regularization

Ask PDF

Apr 18, 2026

CCAR: Intrinsic Robustness as an Emergent Geometric Property

Standard supervised learning optimizes for predictive accuracy but remains agnostic to the internal geometry of learned features, often yielding representations that are entangled and brittle. We propose Class-Conditional Activation Regularization (CCAR) to explicitly engineer the feature space, imposing a block-diagonal structure via a soft inductive bias. By shaping the latent representation to confine class energy to orthogonal subspaces, we create an intrinsic geometric scaffold that naturally filters noise and adversarial perturbations. We provide theoretical analysis linking this structural constraint to the maximization of the Fisher Discriminant Ratio, establishing a formal connection between geometric disentanglement and algorithmic stability. Empirically, this approach demonstrates that robustness is an emergent property of a well-engineered feature space, significantly outperforming baselines on label noise and input corruption benchmarks.

Akash Samanta, Manish Pratap Singh, Debasis Chaudhuri

Techno India University, Salt Lake, Kolkata, India · DRDO Young Scientist Laboratory - CT, Chennai, India

Disentangled Representation Learning Representation Learning Algorithmic Stability Neural Network Robustness Adversarial Robustness Orthogonal Representation Learning+3

Ask PDF

Abstract

Active Learning

Explore similar work

Neural Collapse by Design: Learning Class Prototypes on the Hypersphere

Human-Centered Learning Mechanics: Observable Entropy Control under Protected-Risk Constraints

CCAR: Intrinsic Robustness as an Emergent Geometric Property