cs.CVFeb 20, 2026

Self-Aware Object Detection via Degradation Manifolds

Authors: Stefan Becker, Simon Weiss, Wolfgang Hübner, Michael Arens

Organizations: Fraunhofer Institute of Optronics, System Technologies and Image Exploitation IOSB Gutleuthausstraße 1, 76275 Ettlingen

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Object detectors achieve strong performance under nominal imaging conditions but can fail silently when exposed to blur, noise, compression, adverse weather, or resolution changes. In safety-critical settings, it is therefore insufficient to produce predictions without assessing whether the input remains within the detector's nominal operating regime. We refer to this capability as self-aware object detection. We introduce a degradation-aware self-awareness framework based on degradation manifolds, which explicitly structure a detector's feature space according to image degradation rather than semantic content. Our method augments a standard detection backbone with a lightweight embedding head trained via multi-layer contrastive learning. Images sharing the same degradation composition are pulled together, while differing degradation configurations are pushed apart, yielding a geometrically organized representation that captures degradation type and severity without requiring degradation labels or explicit density modeling. To anchor the learned geometry, we estimate a pristine prototype from clean training embeddings, defining a nominal operating point in representation space. Self-awareness emerges as geometric deviation from this reference, providing an intrinsic, image-level signal of degradation-induced shift that is independent of detection confidence. Extensive experiments on synthetic corruption benchmarks, cross-dataset zero-shot transfer, and natural weather-induced distribution shifts demonstrate strong pristine-degraded separability, consistent behavior across multiple detector architectures, and robust generalization under semantic shift. These results suggest that degradation-aware representation geometry provides a practical and detector-agnostic foundation.

Figures & tables

Figure 1: Proposed degradation-manifold framework for self-awareness. Multi-layer backbone features are fused via 1×11{\times}1 projections and attention pooling and mapped into a normalized embedding space trained with contrastive degradation compositions. A pristine prototype, computed from clean training images, anchors the manifold to nominal operating conditions. At inference, cosine distance to this prototype yields the image-level degradation score Sdeg(x)S_{\mathrm{deg}}(\mathbf{x}) .
Figure 2: Detection performance ([email protected] and [email protected]) of different detection models under increasing corruption severity on the COCO val2017 dataset. Here the results of YOLOv9-m Wang et al. (2024b) , YOLOv10-m Wang et al. (2024a) , YOLOv11-m Jocher and Qiu (2024) , and the transformer-based RT-DETR-l Zhao et al. (2024) are shown.
Figure 3: Relative detection performance drop of YOLOv10-m Wang et al. (2024a) on COCO val2017 under increasing native severity levels ([email protected]:0.95). Each cell shows the percentage drop relative to the clean val2017 baseline (pristine COCO images, no added degradations). Brighter cells indicate larger drops. Left: degradations from Michaelis et al. (2019) . Right: degradations from Agnolucci et al. (2024) .
Figure 4: t-SNE visualization Hinton and Roweis (2003) of the learned degradation manifold under two corruption taxonomies. ( Top ) Degradations from Michaelis et al. (2019) based on Hendrycks and Dietterich (2019) . ( Bottom ) Degradations from Agnolucci et al. (2024) . Each color denotes a specific degradation type, while marker styles indicate the corresponding degradation group. For visualization, 100 COCO Lin et al. (2014) samples are corrupted at severity level 5. Distinct clusters emerge for different degradation types. For training, only the degradations from Agnolucci et al. (2024) were used.
Figure 5: t-SNE visualization Hinton and Roweis (2003) of the learned degradation manifold. Samples are drawn from multiple datasets (COCO Lin et al. (2014) , BDD Yu et al. (2020) , KITTI Geiger et al. (2012) , DETRAC Wen et al. (2020) , UAVDT Du et al. (2018) , and FLIR (VIS) FLIR (2022) (V2) ). Degradations are clearly separated in embedding space, while pristine images form a compact cluster across datasets, indicating content-independence of the learned representation. Notably, BDD and KITTI images were not used during training, demonstrating cross-dataset generalization of the degradation manifold and that the embedding organizes images according to degradation characteristics rather than semantic content.
 Severity levels AUROC ↑\uparrow
Model12345
Image Quality Assessment (IQA)     
MANIQA Yang et al. (2022) (score-based)52.8854.4955.2756.6761.56
MANIQA Yang et al. (2022) (embedding-based; prototype anchored)65.8771.5573.4276.9777.30
QualiCLIP Agnolucci et al. (2025) (score-based)53.1756.7766.2870.6072.53
QualiCLIP Agnolucci et al. (2025) (embedding-based; prototype anchored)45.3641.5346.7749.7956.16
Table 1: AUROC scores of detecting degradation-induced covariate shift for synthetically corrupted variants of the COCO dataset. Best results are highlighted as first , second , and third .
Degradation Manifold (Ours)
 Severity levels AUROC ↑\uparrow
Dataset12345
VISDRONE Zhu et al. (2021)83.6689.1592.0895.8596.79
KITTI Geiger et al. (2012)94.1996.3797.1697.2598.52
DETRAC Wen et al. (2020)94.6597.0598.8799.6999.81
FLIR (VIS) FLIR (2022) (V2)82.8391.3992.7295.1196.66
Table 2: AUROC scores of detecting OoD covariate shift using our degradation manifold across different datasets.
Degradation Manifold (Ours)
 Severity levels AUROC ↑\uparrow
Datasets12345
COCO Lin et al. (2014) & VISDRONE Zhu et al. (2021)83.5489.9190.0395.0695.95
COCO Lin et al. (2014) & KITTI Geiger et al. (2012)89.2792.4493.8696.4097.21
COCO Lin et al. (2014) & DETRAC Wen et al. (2020)92.2693.2994.9298.6298.97
COCO Lin et al. (2014) & FLIR (VIS) FLIR (2022) (V2)82.4588.9289.3294.3295.97
Table 3: AUROC for pristine–degraded separability under mixed-dataset evaluation. Pristine samples are drawn from combined datasets, while degraded samples originate from a degraded version of the datasets. COCO is subsampled for class-balanced comparison. Results demonstrate stable degradation-aware separability under simultaneous content and degradation variation.
Table 4: Natural weather-induced distribution shifts on BDD Yu et al. (2020) and STF Bijelic et al. (2020) . AUROC measures separability between nominal-weather (pristine) (ID) and adverse-weather (OoD) subsets. Training with synthetic weather corruptions improves transfer to real-world heavy weather conditions.
Degradation Manifold (Ours)
 Severity levels AUROC ↑\uparrow
Variant12345
Full model (multi-layer + attention + hard neg.)88.6489.7089.7595.2897.14
Last backbone layer only81.8785.1387.5689.1392.75
Global average pooling (no attention)85.5188.4489.4995.0196.30
Without hard negative mining86.0087.1787.5890.4191.54
Table 5: Ablation study of the proposed degradation manifold using a YOLOv10-m backbone. We analyze the impact of multi-layer readout, attention-based pooling, and hard negative mining on AUROC across severity levels. The full configuration consistently achieves the strongest separability.
Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Examples from Seeing Through Fog (STF) Bijelic et al. (2020) used for the natural weather-shift evaluation. We construct an ID split from clear/nominal scenes and OoD splits from adverse-weather conditions ( Dense Fog , Heavy Snow , and Heavy Rain after filtering).
Figure 7: Examples from BDD100K Yu et al. (2020) illustrating the weather-based ID/OoD construction. Left: ID example from the clear subset. Middle: ambiguous rainy example with weak cues (e.g., wet road only), which is excluded by manual refinement. Right: retained OoD example with strong visible rain cues ( heavy rain ).
Figure 8: t-SNE visualization Hinton and Roweis (2003) of the learned degradation manifold under the corruption taxonomy of Michaelis et al. (2019) , based on Hendrycks and Dietterich (2019) , shown for three severity levels (1, 3, and 5). Each color denotes a specific degradation type, while marker styles indicate the corresponding degradation group. For visualization, 100 COCO Lin et al. (2014) samples are corrupted at the respective severity level. At low severity (level 1), clusters begin to form but remain partially overlapping. With increasing severity (levels 3 and 5), degradation types become progressively more separated, indicating that the embedding geometry preserves a meaningful ordering with respect to degradation strength. Note that training used degradation compositions from Agnolucci et al. (2024) , demonstrating zero-shot generalization of the learned manifold.
Figure 9: t-SNE visualization Hinton and Roweis (2003) of the learned degradation manifold under the corruption taxonomy of Agnolucci et al. (2024) , shown for three severity levels (1, 3, and 5). Each color denotes a specific degradation type, while marker styles indicate the corresponding degradation group. For visualization, 100 COCO Lin et al. (2014) samples are corrupted at the respective severity level. At low severity (level 1), clusters begin to form but remain partially overlapping. With increasing severity (levels 3 and 5), degradation types become progressively more separated, indicating that the embedding geometry preserves a meaningful ordering with respect to degradation strength.
   Severity levels AUROC ↑\uparrow
 DetectorModel12345
Top- 33 Detection UncertaintiesFasterRCNN DMMTrace51.2253.4954.8457.7562.91
Determinant48.6347.1547.1847.4251.31
Entropy (Regression)50.7152.5553.8557.7762.40
Confidence Score53.5159.0460.5067.9171.23
Entropy (Classification)55.8259.3063.7670.1774.93
Appendix
Table 6: AUROC scores for Top- 33 detection uncertainties (as proposed from Oksuz et al. (2023) ) across using the probabilistic detector of Harakeh and Waslander (2021) on synthetically corrupted versions of the COCO dataset.
 Severity Levels AUROC ↑\uparrow
Degradation Function Agnolucci et al. (2024)12345
’Gaussian Blur’48.1561.5685.2099.8499.89
’Lens Blur’98.0298.4598.9099.2099.50
’Motion Blur’48.9393.4399.6299.9299.95
’White Noise’98.3899.2899.6099.7599.85
’White Noise in Color Component’85.2398.7299.3699.7099.85
Appendix
Table 7: AUROC for pristine–degraded separability under individual synthetic degradations on COCO. Degradations follow the IQA-style definitions of Agnolucci et al. (2024) .
 Severity Levels AUROC ↑\uparrow
Degradation Function Hendrycks and Dietterich (2019)12345
’Gaussian Noise’99.9199.9499.9399.9499.92
’Shot Noise’99.6399.7499.8399.8199.85
’Impulse Noise’99.9699.9499.9299.9199.90
’Speckle Noise’99.7099.7299.7399.7299.71
’Defocus Blur’99.8699.8799.8899.8799.86
Appendix
Table 8: AUROC for pristine–degraded separability under individual synthetic degradations on COCO. Degradations follow the robustness benchmarks of Hendrycks and Dietterich (2019) and Michaelis et al. (2019) .
Figure 10: Distribution of Sdeg(x)S_{\mathrm{deg}}(\mathbf{x}) scores for ID and OoD samples at different corruption severity levels, using degradation manifold approach with a YOLOv10-m backbone.
Figure 11: Detection performance of YOLOv10-m Wang et al. (2024a) under different training objectives on the COCO-mini person subset Samet et al. (2020) . Detection-only training preserves baseline accuracy. Degradation-only training significantly disrupts semantic detection features. Joint multi-task optimization mitigates this effect while introducing a moderate performance trade-off.
Figure 12: Overview of the applied image degradation model. Randomly assembled degradation compositions CC are applied sequentially to synthesize diverse degradation patterns. Each degradation composition contains up to NdegN_{\text{deg}} degradations sampled from pre-defined degradation groups (e.g., the degradation groups of Agnolucci et al. (2024) ).
Figure 13: Training pair construction for degradation-manifold learning. Two pristine images are degraded using the same sampled composition, forming a positive pair in embedding space ( z1,z2z^{1},z^{2} ). To construct hard negatives, we apply an additional half-size crop before resizing to the detector input resolution. All degraded views are resized to the detector’s default input size prior to encoding. The objective pulls embeddings of equally degraded full-resolution images together while pushing them away from their resolution-perturbed counterparts, encouraging sensitivity to fidelity changes beyond semantic similarity.
More in this topic

Object Detection

Object detection locates and classifies objects in images or video, balancing localization accuracy, robustness, and inference speed.

See all 624 papers

Explore similar work

CardsList
Oct 3, 2026

Task-Sensitive Geometry of Representation Transfer for Object Detection under Image Degradation

Object detection under image degradation can benefit from clean-image supervision, but aggregate gains do not imply that transferred representation changes are uniformly useful. We study how clean task knowledge affects degraded-image representations and whether local responses to structured representation directions can be characterized geometrically. Using paired clean and Gaussian-degraded BDD100K images, we show that clean-teacher distillation improves observed detection accuracy while producing heterogeneous object-level transfer. We isolate a representation component complementary to direct clean-teacher alignment and map it into the distilled student space through an orthogonal bridge. Controlled interventions rescue 13.22% of objects lost under the distilled representation, versus 4.30% under norm-matched random perturbations, with very low harm on preserved objects. We introduce task-sensitive geometry, a gradient-derived channel-space geometry constructed from normalized detection-loss gradients. On a reserved cohort, mapped-complement orientation within this frozen geometry is positively associated with local intervention-response magnitude after controlling for intervention magnitude (partial Spearman ρρ = 0.242, 95% CI [0.108, 0.359]). The relationship eplicates on independent data (ρρ = 0.180) and with RT-DETR-L (ρρ = 0.227), but not for the direct clean-teacher residual family, and it weakens for large interventions. Routing rules and specialized distillation objectives based on these signals do not yield statistically reliable gains over CLEANKD. These results support a local, direction-family-dependent task-sensitive geometry while showing that converting such structure into improved global training remains an open problem.

Van Vung Pham

Computer Science Department, Sam Houston State University, Huntsville, Texas, USA

Object DetectionFeature-Based Knowledge Distillation
AskPDF
Apr 11, 2026

Degradation-Consistent Paired Training for Robust AI-Generated Image Detection

AI-generated image detectors suffer significant performance degradation under real-world image corruptions such as JPEG compression, Gaussian blur, and resolution downsampling. We observe that state-of-the-art methods, including B-Free, treat degradation robustness as a byproduct of data augmentation rather than an explicit training objective. In this work, we propose Degradation-Consistent Paired Training (DCPT), a simple yet effective training strategy that explicitly enforces robustness through paired consistency constraints. For each training image, we construct a clean view and a degraded view, then impose two constraints: a feature consistency loss that minimizes the cosine distance between clean and degraded representations, and a prediction consistency loss based on symmetric KL divergence that aligns output distributions across views. DCPT adds zero additional parameters and zero inference overhead. Experiments on the Synthbuster benchmark (9 generators, 8 degradation conditions) demonstrate that DCPT improves the degraded-condition average accuracy by 9.1 percentage points compared to an identical baseline without paired training, while sacrificing only 0.9% clean accuracy. The improvement is most pronounced under JPEG compression (+15.7% to +17.9%). Ablation further reveals that adding architectural components leads to overfitting on limited training data, confirming that training objective improvement is more effective than architectural augmentation for degradation robustness.

Zongyou Yang, Yinghan Hou, Xiaokun Yang

Department of Earth Science and Engineering, Imperial College London, London, United Kingdom · School of Electronic Information, Nanchang Institute of Technology, Nanchang, China

AI-Generated Image DetectionImage Corruption RobustnessConsistency Regularization
AskPDF
Jun 29, 2026

FR-DETR: Frequency and Recurrent Feature Refinement for Robust Object Detection under Adverse Weather

Object detection under adverse weather remains challenging due to severe visual degradations and domain shifts. Existing enhancer-based approaches attempt to improve detection by cascading an enhancer with a detector, but they introduce redundant feature extraction and incur high computational cost with limited accuracy gains when paired with SOTA detectors. We propose FR-DETR, a detector-centric framework that refines features rather than images, focusing enhancement on regions of interest and leveraging frequency-domain cues. Specifically, we design (I) a Frequency Refinement Module that dynamically separates and reweights low- and high-frequency components to improve foreground-background discrimination, and (II) a Recurrent Focus Refinement Module (RFRM) that iteratively refines features using coarse predictions as guidance. Extensive experiments demonstrate that FR-DETR achieves superior detection accuracy under adverse weather while being significantly more computationally efficient than enhancer-based methods. Our implementation is available at https://github.com/ducnt1210/FR-DETR.

Tuan-Duc Nguyen, Duc-Trong Le

FPT Software AI Center · VNU University of Engineering and Technology

Frequency-Domain Feature LearningDetection TransformerObject Detection
AskPDF