cs.CVNov 9, 2026

Aggregation-Induced Calibration Failure in Sum-Based Label Expansion for Multi-Label Facial Expression Recognition: A Diagnostic Study

Authors: Weihang Yuan

Organizations: Beijing Forestry University, Beijing, China

Abstract

Label-space expansion - predicting a larger joint space and summing back to the original labels - is a natural design for multi-label facial expression recognition, but its failure modes have not been characterised. We evaluate Condition-Aware Label Expansion (CALE) on a self-built dataset (30,240 training images, 3,460 multi-label test images), on seven backbones, and on the public RAF-DB and RAF-ML benchmarks. The central finding is that the aggregation rule, not the representation, decides whether the method works. With a fixed threshold of 0.5, the aggregate score spans 28x across five backbones at 10% of the training data (0.24 to 6.79) and collapses to a 1.9x range at full data, while threshold-free mAP stays within 1.28x over the same backbones, which localises the fault to the summation step. The mechanism is the uncalibrated aggregation rule, which adds the main-branch score to half the sum of the three non-main branches; we verify it by measuring main- and non-main-branch activations. Two plain seven-dimensional sigmoid baselines (BCE, ASL) reach mAP 0.5055 and 0.5175; ASL exceeds three of the five CALE configurations and falls within 0.03 of the best. On single-label FER the gain over a seven-dimensional baseline is nominally significant (paired t-test, p = 0.0436) but does not survive Benjamini-Hochberg correction (p_BH = 0.2055) or a Wilcoxon test (p = 0.1250). We conclude that sum-based label expansion requires threshold calibration at the deployment data scale, or a learned aggregation, before it can support multi-label decisions.

Figures & tables

Explore similar work

CardsList
  1. Predicting Human Disagreement for Calibrated Dynamic Facial Expression Recognition

    Sep 15, 2026Yiming Wang, Frederick W. B. Li, Jingyun WangAnnotator DisagreementUncertainty Quantification

  2. How to Estimate Whether You Have Found Several Needles in a Haystack: Measuring Calibration in Multi-Label Text Classification

    Sep 22, 2026Sophie Henning, Georg Hofmann, Alexander Schulte +2Multi-Label ClassificationLanguage Model Calibration

  3. Multi-Label Agreement: A Flexible Framework for Inter-Annotator Reliability

    Oct 27, 2026Tim Menzner, Jochen L. LeidnerInter-Rater Reliability