Not All Confusion Is Equal: A Source-Aware Uncertainty Diagnosis for Fine-Grained Aircraft Detection

Hai Huang; Helmut Mayer

cs.CVSep 24, 2026

Not All Confusion Is Equal: A Source-Aware Uncertainty Diagnosis for Fine-Grained Aircraft Detection

Authors: Hai Huang, Helmut Mayer

Organizations: Chair of Visual Computing, Institute for Applied Computer Science, Universität der Bundeswehr München, Germany

Abstract

Fine-grained object detectors are commonly evaluated with confusion matrices, which show where the model is confused but not why, nor whether the confusion can be reduced. We argue that confusion can be attributed to distinct, separable sources, each quantitatively measurable, turning a passive measurement into actionable guidance. We present $A^2E^2$ , a diagnostic tool that decomposes the sources of confusion along two axes, $\{$ aleatoric, epistemic $\} \times \{$ within-class, between-class $\}$ , giving a $2\times2$ taxonomy that enumerates the source types. Each quadrant is measured by its own quantity, computed in one of three places (input geometry, output-space disagreement, and the bias-parameter posterior), so the two epistemic sources are separated by construction rather than by an empirical correlation. On fine-grained aircraft detection, the four quadrants become four named sources with their own remedy verdict: affinity (geometric similarity, irreducible from size alone), heterogeneity (geometrically heterogeneous sub-variants, pointing to re-labeling rather than more data), contested (an insufficiently trained but learnable boundary, improvable), and collapsed (a class starved of data, reducible). After attributing the confusion to a specific reducible source, we apply a targeted intervention and verify experimentally that it reduces the diagnosed source specifically while leaving the irreducible sources unchanged. $A^2E^2$ thus turns confusion measurement into a concrete, validatable and actionable "diagnosis" in which the same off-diagonal mass can carry opposite causes and opposite remedies. We also state this framework's limits, including which sources are only partially identifiable on this specific dataset and why.

Figures & tables

Figure 1: Overview of $A^{2}E^{2}$ : the four confusion sources as matrices over the ten classes. Within-class sources occupy the diagonal only, between-class sources the (symmetric) upper triangle. Affinity (between-ale) is the OBB-size Bhattacharyya overlap BC ( Eq. 2 ); contested (between-epi) is the per-pair mutual information MI ( Eq. 4 ); collapsed (within-epi) is the bias-posterior variance $\sigma^{2}_{b}$ ( Eq. 5 ); heterogeneity (within-ale) is the size dispersion (CV) over a noise floor, split per diagonal cell into body/wing (collapsed classes shown but confounded by scarcity, cf. Fig. 3 ). Aleatoric sources in orange, epistemic in purple. Red dots mark MI pairs involving ARJ21 or Boeing777 (zero top-1 predictions at baseline), computed on a restricted population ( Sec. 3.3 ) and not read as contested.

Case	metric	value	Attributed source	Remedy verdict
ARJ21 (class)	$\sigma^{2}_{b}$	rank 1/10	collapsed (within-epi)	add class data (verified, T1)
A220–A350 (pair)	MI	$0.387$	contested (between-epi)	add boundary data (supported, directional, T2)
A330–Boeing787 (pair)	BC	$0.945$	affinity (between-ale)	irreducible from size; more data will not help
A330 (class)	excess-CV	$+0.011$ body	heterogeneity (within-ale)	more data will not help; re-label (initial test null)

Table 1: Four study cases with remedy verdict covering the four quadrants

Metric	Predicted	Observed	Verdict
val AP	$\uparrow$	$0.000\rightarrow 0.333$	$\checkmark$ strong
test AP	$\uparrow$	$0.000\rightarrow 0.139$	$\checkmark$ (test harder than val)
$\sigma^{2}_{b}$	$\downarrow$	$-12.5\%$	$\checkmark$ targeted narrowing
retention (test)	$\uparrow$	$0.000\rightarrow 0.072$	$\checkmark$ targeted revival
$C_{\textrm{ale}}$ ref.	no change	$\approx-0.04$ (uniform)	no targeted response
BC	no change	unchanged	geometry is data-independent

Table 2: Verification of T1 (ARJ21 collapsed-epistemic intervention): pre-registered predictions vs. observed outcomes. All four predictions on ARJ21 are confirmed; the aleatoric-reference rows show no matching targeted response. The untreated collapsed control (Boeing777) is discussed in the text. See Sec. 4.5 .

Metric	Predicted	Observed	Verdict
A220–A350 MI	$\downarrow$	$0.387\rightarrow 0.183$ ( $-53\%$ )	$\checkmark$ strong
A350 val AP	$\uparrow$	$0.698\rightarrow 0.788$ ( $+0.090$ )	$\checkmark$
A350 test AP	$\uparrow$	$0.582\rightarrow 0.619$ ( $+0.037$ )	$\checkmark$
BC (overlap)	unchanged	replicated from baseline	$\checkmark$ by construction
A220 val AP	— (not predicted)	$+0.027$	neutral-to-positive
A220 test AP	— (not predicted)	$-0.041$	opposite sign from val

Table 3: Verification of T2 (A220–A350 contested-epistemic intervention): pre-registered predictions vs. observed outcomes. The main effect (MI drop, A350 improvement, BC unchanged) is confirmed on both splits; A220’s val/test divergence was not predicted and is a caveat discussed in the text.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 2: Per-class physical size geometry (body length vs. wingspan, meters, GSD-corrected) on the development set, with Gaussian $1\sigma/2\sigma$ contours. Narrow- and wide-body aircraft are well separated (BC $\approx 0$ ).

Measure (source)	ARJ21	Reading
$\sigma^{2}_{b}$ (collapsed, Laplace bias posterior)	$6.48\times 10^{-3}$	rank 1 of 10
retention ratio (collapsed, raw counts)	$0.000$	tied-lowest
MI (“other”–ARJ21) (contested, output space)	$0.084$	low-to-moderate

Appendix

Table 4: Single-class cross-validation on ARJ21, the rarest class. Two measures computed along different paths (Laplace posterior and prediction counts) both read collapsed-extreme, while the output-space contested measure reads low; see Appendix 0.C.3 .

Figure 3: Heterogeneity metric: class-conditional size CV against the noise floor (dashed, mean CV of the reference classes A321/A350, $\approx 0.062$ ; A350 is drawn as multi-variant by catalogue, but its instances here are predominantly one variant, Sec. 3.3 ). Left : body length; the multi-variant classes B737/A330/A220/B787 exceed the floor, consistent with their sub-variants; single-variant A321 (gray) sits at it. Right : wingspan; the same classes sit near the floor, since their variants share a wingspan. Collapsed classes (orange, ARJ21/B777) are excluded from anchoring: their dispersion is confounded by scarcity and, for B747/B777 wingspan, by OBB artifacts. See Sec. 5.1 .

Figure 4: Boeing777 body length shows a genuine bimodal structure whose peak spacing ( $\approx 11$ m) matches the 777-200/-300 fuselage difference ( $10.2$ m). However, Boeing777 is a collapsed class, so this within-class signal is confounded with data scarcity and we do not use it as a heterogeneity anchor; it is shown here as a rejected/contaminated case.

Explore similar work

Hierarchical Fine-Grained Aerial Object Detection
Jun 15, 2026Yan Zhang, Fang Xu, Wen Yang +1Aerial Dataset Fine-Grained Perception
School of Computer Science, Wuhan University, Wuhan, 430072, China · School of Artificial Intelligence, Wuhan University, Wuhan, 430072, China · School of Electronic Information, Wuhan University, Wuhan, 430072, China
Differences in Detection: Explainability Where it Matters
Jun 5, 2026Johannes Theodoridis, Johannes Maucher, Andreas SchillingDetection Performance Open-Vocabulary Object Detection
University of T ubingen · Institute for Applied AI · Hochschule der Medien Stuttgart
Diagnosing Aerial-View Object Detectors with Foundational Image Generative Models
Jul 2, 2026Stanislav Panev, Minhyek Jeon, Vaishnavi Khindkar +5Aerial Dataset Synthetic Image Detection
Carnegie Mellon University, USA · DEVCOM Army Research Laboratory, USA · Florida State University, USA

Jun 15, 2026, cs.CVJ/K move · Enter open · S save

Hierarchical Fine-Grained Aerial Object Detection

Yan Zhang, Fang Xu, Wen Yang, Gui-Song Xia

School of Computer Science, Wuhan University, Wuhan, 430072, China · School of Artificial Intelligence, Wuhan University, Wuhan, 430072, China · School of Electronic Information, Wuhan University, Wuhan, 430072, China

Aerial Dataset Fine-Grained Perception Scene Reasoning

Open paper AskPDF arXiv

Fine-grained aerial object detection, driven by the intrinsic granularity of real-world object categories, is crucial for advanced scene understanding in remote sensing. Existing methods largely inherit the paradigm of coarse-grained object detection, relying solely on single-label supervision and thus struggling to distinguish model-level categories with subtle structural differences. However, for each specific model (e.g., Boeing 787), structured prior knowledge such as attributes and hierarchies offers discriminative semantics across multiple granularities. Motivated by this, we present ExpertDet, a scheme that incorporates expert-informed cues to enhance fine-grained aerial object detection. Specifically, we design Vision-aware Masked Attribute Modeling (VMAM), which aligns attribute semantics with visual structures by reconstructing randomly masked attributes from visual cues, enabling the detector to capture subtle structural distinctions. We further propose Hierarchical Visual Instance Promotion (HierVIP), which builds a visual prototype tree based on hierarchical relations and imposes taxonomy-aware constraints to preserve cross-level semantic continuity while enhancing category discrimination. Moreover, we curate a new fine-grained object detection benchmark for Precise recognition of model-specific Ships and Planes from aerial imagery, PSP, covering 106 ship classes and 30 airplane models, respectively, featuring the most extensive collection of model-specific categories among existing aerial object detection datasets to date. We benchmark state-of-the-art object detection algorithms on the PSP benchmark. Extensive evaluation demonstrates that ExpertDet consistently outperforms other fine-grained competitors across hierarchy levels. The dataset, benchmark, and code are available at https://nnnnerd.github.io/PSP-Benchmark/.

Abstract

Explore similar work

Hierarchical Fine-Grained Aerial Object Detection

Differences in Detection: Explainability Where it Matters

Diagnosing Aerial-View Object Detectors with Foundational Image Generative Models