Existing neural network calibration methods often treat calibration as a static, post-hoc optimization task. However, this neglects the dynamic and temporal nature of real-world inference. Moreover, existing methods do not provide an intuitive interface enabling human operators to dynamically adjust model behavior under shifting conditions. In this work, we propose Knob, a framework that connects deep learning with classical control theory by mapping neural gating dynamics to a second-order mechanical system. By establishing correspondences between physical parameters -- damping ratio (ζ) and natural frequency (ωn) -- and neural gating, we create a tunable "safety valve". The core mechanism employs a logit-level convex fusion, functioning as an input-adaptive temperature scaling. It tends to reduce model confidence particularly when model branches produce conflicting predictions. Furthermore, by imposing second-order dynamics (Knob-ODE), we enable a dual-mode inference: standard i.i.d. processing for static tasks, and state-preserving processing for continuous streams. Our framework allows operators to tune "stability" and "sensitivity" through familiar physical analogues. This paper presents an exploratory architectural interface; we focus on demonstrating the concept and validating its control-theoretic properties rather than claiming state-of-the-art calibration performance. Experiments on CIFAR-10-C validate the calibration mechanism and demonstrate that, in Continuous Mode, the gate responses are consistent with standard second-order control signatures (step settling and low-pass attenuation), paving the way for predictable human-in-the-loop tuning.
Figures & tables
Figure 1: The Knob framework as a physics-inspired interface. Left (Dual-Stream Backbone): A shared encoder feeds two lightweight projection heads, producing a robust “Static” branch and a more sensitive “Dynamic” branch, yielding complementary logit vectors zstatic and zdyn . Center (Physics Engine): The gating mechanism is modeled as a mass-spring-damper system (normalized to unit mass). A network-predicted reference input u∗(x) drives the system; its response is governed by two interpretable control parameters— Natural Frequency ( ωn ), controlling sensitivity/bandwidth, and Damping Ratio ( ζ ), controlling stability/conservativeness—which act as tunable “knobs” for the operator. Right (Convex Fusion): The gate value g(x)∈[0,1] , determined by the mass displacement, performs a convex combination of two logit branches, naturally limiting model overconfidence.
Symbol/Term
Definition and Intuition
Methodology Symbols
zstatic,zdyn
Logit vectors from the static and dynamic branches, respectively.
g(x)
A sample-wise scalar gate value in [0,1] that controls the convex fusion.
zfuse
The final logit vector after convex fusion.
Teff(x)
The effective temperature; a measure of induced smoothing ( Teff≥1 ).
ζ,ωn
Damping ratio and natural frequency; control overshoot and convergence speed.
Table 1: Terminology and Metric Definitions.
Method
Avg-C (%) ↑
Err-C (%) ↓
ECE deb ↓
Latency (ms)
Static
71.25
28.75
0.1553
3.48
Attention
70.45
29.55
0.1648
3.51
Knob-IA
72.52
27.48
0.1617
3.49
ODE-Lite
71.62
28.38
0.1488
3.64
Knob-ODE
71.49
28.51
0.1829
4.41
Table 2: Main results on CIFAR-10-C (Reset Mode). Calibration and efficiency under identical budgets (3-run averages). Lower is better for Err-C and ECE deb . Avg-C = mean accuracy averaged over severities and corruptions.
Figure 2: Cost–calibration Pareto frontiers. Left: ECE deb vs. GFLOPs. Right: ECE deb vs. latency. ODE-Lite lies near the frontier in both views, indicating a favorable calibration–cost trade-off.
Method
Params (M)
GFLOPs
Latency (ms)
Static
11.70
0.5584
3.48
Attention
11.90
0.5586
3.51
Knob-IA
11.77
0.5585
3.49
ODE-Lite
11.77
0.5585
3.64
Knob-ODE
11.77
0.5585
4.41
Table 3: Efficiency comparison. The gate controller introduces only O(D) incremental overhead, and overall computational costs are essentially identical across methods.
Figure 3: E-1: Static equivalence evidence (Knob-IA). (a) CSR for the overall, disagreement, and agreement subsets stays below 1 and decreases with corruption severity. (b) T(x)≥1 and increases with severity (Spearman’s ρ=0.6 ). (c) Reliability diagram comparing disagreement and agreement subsets. (d) Per-bin ECE by confidence bins.
Figure 4: E-2: Learning-level probe—gate “taking sides.” Treating g(x) as a “dynamic-preferred” score, we plot AUC for Oracle Advantage (OA, solid lines) and Gradient-Consistent Advantage (GCA, dashed lines) across different sample subsets (overall, top-1 disagreement, margin sign). The polarization index Π=E[∣g−0.5∣] is shown on the right axis (gray triangles). AUC steadily increases during training, with overall AUCOA≈0.79 and AUCGCA≈0.74 by epoch 50, while Π remains low ( ≈0.10 ), indicating the gate learns to discriminate without saturating.
Figure 5: E-3: Dynamics-level probe (Continuous Mode). (a) Step Response: Under a shot-noise severity schedule ( 1→5→1 ), the gate g(t) (blue) exhibits a smooth, damped transition rather than an instantaneous jump. Shaded bands indicate the inter-quartile range (IQR) over stochastic corruption realizations. (b) Empirical Bode Plot: The magnitude response relative to DC (0 dB) shows a downward trend as frequency increases, consistent with soft low-pass filtering behavior. Error bars represent standard error of the mean (SEM).
Neural networks (NNs), while highly flexible and versatile, are often poorly calibrated. Post-hoc calibration methods adjust the outputs of an already trained model using a separate calibration dataset and additional calibration parameters. This process adds overhead that requires separate optimization of the calibrator and subsequent model selection. In this paper, we show that standard (post-hoc) calibrators can be trained end-to-end with the backbone via a bilevel Stackelberg formulation. This formulation eliminates the need for a separate calibration set or a dedicated calibration loss by jointly optimizing accuracy and calibration using cross-entropy, introducing only one additional hyperparameter: the calibrator learning rate. Although our framework supports calibrators ranging from temperature and matrix scaling to NN layers, our results show that using the final classification layer itself as the calibrator yields accurate, well-calibrated models with no additional parameters. Our approach outperforms SOTA calibrators with minimal added complexity, while also avoiding the multiplicative overhead of separate calibration components in settings such as multi-task learning.
Nektarios Kalampalikis, Pablo M. Olmos, Isabel Valera
Dept. of Computer Science Saarland University, Germany
Conventional neural networks strictly separate learning and inference because if parameters are updated during inference, outputs become unstable and even the inference function itself is not well defined [1, 2, 3]. This paper shows that DynamicGate MLP structurally permits learning inference concurrency [4, 5]. The key idea is to separate routing (gating) parameters from representation (prediction) parameters, so that the gate can be adapted online while inference stability is preserved, or weights can be selectively updated only within the inactive subspace [4, 5, 6, 7]. We mathematically formalize sufficient conditions for concurrency and show that even under asynchronous or partial updates, the inference output at each time step can always be interpreted as a forward computation of a valid model snapshot [8, 9, 10]. This suggests that DynamicGate MLP can serve as a practical foundation for online adaptive and on device learning systems [11, 12].
Modern neural networks can achieve high accuracy while remaining poorly calibrated, producing confidence estimates that do not match empirical correctness. Yet calibration is often treated as a post-hoc attribute. We take a different perspective: we study calibration as a training-time phenomenon on small vision tasks, and ask whether calibrated solutions can be obtained reliably by intervening on the training procedure. We identify a tight coupling between calibration, curvature, and margins during training of deep networks under multiple gradient-based methods. Empirically, Expected Calibration Error (ECE) closely tracks curvature-based sharpness throughout optimization. Mathematically, we show that both ECE and Gauss--Newton curvature are controlled, up to problem-specific constants, by the same margin-dependent exponential tail functional along the trajectory. Guided by this mechanism, we introduce a margin-aware training objective that explicitly targets robust-margin tails and local smoothness, yielding improved out-of-sample calibration across optimizers without sacrificing accuracy.