Background-Free Objectness Learning for Class-Agnostic Detection

Dania Batool; Liliana Lo Presti; Marco La Cascia; Filippo Vella

cs.CVAug 29, 2026

Background-Free Objectness Learning for Class-Agnostic Detection

Authors: Dania Batool, Liliana Lo Presti, Marco La Cascia, Filippo Vella

Organizations: Department of Engineering University of Palermo Palermo, Italy · ICAR National Research Council of Italy Palermo, Italy

Abstract

Object detectors are typically trained under closed-set supervision, where unlabeled regions are implicitly treated as background. Under incomplete annotations, this assumption introduces objectness bias: visually valid but unlabeled objects are used as negatives, tying objectness to the annotated taxonomy rather than generic object structure. This limitation is particularly problematic for class-agnostic and open-world detection. This paper proposes Background-Free Objectness Learning (B-FOR), a dense class-agnostic detection framework that learns objectness without explicit background supervision on unlabeled regions. B-FOR formulates detection as the prediction of dense multi-scale object-center and scale fields, from which object hypotheses emerge as local spatial structures. Supervision is confined to reliable annotated regions through spatially structured soft targets, avoiding foreground-background discrimination. To support decoding from emergent local maxima, the paper further introduces displacement-aware scale fields that model object extent as a spatially varying property of the learned objectness field. Experiments on PASCAL VOC, MS-COCO, and Open Images demonstrate strong generalization to unseen categories and cross-dataset object distributions. B-FOR improves recall by more than +10 AR points over prior class-agnostic baselines. Ablation studies show that both localized objectness supervision and displacement-aware scale fields are critical for class-agnostic localization under incomplete annotations. Code available at: https://github.com/Daniaawan/B-FOR.

Figures & tables

Figure 1: (a) Detection datasets are often incompletely annotated: green boxes indicate labeled objects; red boxes indicate unlabeled object-like regions. (b) B-FOR predicts dense multi-scale object-center and scale fields. (c) Predicted responses consistently activate on both annotated and unlabeled object-like regions across small, medium, and large scales.

Figure 2: Overview of B-FOR. Given an input image, cascaded residual and feature pyramid networks generate multi-scale feature maps that are decoded by lightweight heads $D_{\ell}$ into dense object representations: an object-center score field $\hat{s}_{\ell}$ and center-conditioned width/height fields $\hat{B}^{x}_{\ell}$ and $\hat{B}^{y}_{\ell}$ across small, medium, and large object scales. During inference, local maxima of the score fields define candidate centers, bounding boxes are recovered from the corresponding scale fields, and NMS produces the final class-agnostic detections.

Figure 3: Decoder head for pyramid level

\ell

. Feature maps are upsampled to the prediction grid and decoded into dense outputs: an object-center score field

\hat{s}_{\ell}

, and normalized width and height fields,

\hat{B}^{x}_{\ell}

and

\hat{B}^{y}_{\ell}

	$\displaystyle w_{q}$	$\displaystyle=$	$\displaystyle w_{n}+2\|x_{n}-u\|,$		(8)
	$\displaystyle h_{q}$	$\displaystyle=$	$\displaystyle h_{n}+2\|y_{n}-v\|.$		(9)

Figure 4: The green box denotes the annotated object box

b_{n}=(x_{n},y_{n},w_{n},h_{n})

. When a box is decoded at

q=(u,v)

and must enclose the object, its extent expands symmetrically to compensate for the offset. The orange box, centered at

q

, has width

w_{q}

and height

h_{q}

Table 5

	VOC 20 $\rightarrow$ COCO 60				COCO 80 $\rightarrow$ OI 490
Model	$AR_{Ovr}$	$AR_{Sml}$	$AR_{Med}$	$AR_{Lrg}$	$AR_{Ovr}$	$AR_{Sml}$	$AR_{Med}$	$AR_{Lrg}$
CenterNet † [ 36 ]	17.8	6.6	17.8	41.4	37.4	9.0	25.7	48.4
FCOS † [ 26 ]	NA	NA	NA	NA	41.3	7.5	24.6	55.4
FRCNN-aw [ 8 ]	12.4	1.8	11.0	37.0	17.0	1.6	8.3	33.0
FRCNN-ag [ 8 ]	13.6	2.4	13.1	38.0	18.2	2.8	12.5	31.8
SSD-ag [ 8 ]	18.1	3.1	23.9	40.4	20.6	2.3	20.6	31.4

Table 3: Cross-dataset unseen-category evaluation. AR@1000 at IoU

\geq 0.5

; all methods evaluated class-agnostically. FRCNN, SSD, and UniDetector results are from [ 8 , 31 ] ;

\dagger

own evaluation on public checkpoints (CenterNet DLA-34, FCOS ResNet-50), included as class-supervised references, not matched baselines. FCOS VOC20

\to

COCO60 unavailable: no public VOC-trained checkpoint. B-FOR trains from scratch without class-level supervision.

Figure 5: Objectness score distributions at annotated ( labeled , green) and unannotated ( unlabeled , red) object centers.

y

-axis: probability density, bin width

=0.04

. Dashed lines mark group means;

\Delta=\hat{s}_{\text{lab}}-\hat{s}_{\text{unlab}}

quantifies objectness bias. CenterNet and FCOS both exhibit a sharp spike near zero for unlabeled objects reflecting suppression of unlabeled regions through foreground–background supervision. In contrast, B-FOR yields a broader unlabeled distribution with substantial overlap between labeled and unlabeled objects.

Table 8

Figure 6: Top-1 CAOD on COCO val (VOC20-trained model). Many top-1 predictions localise visually coherent but unannotated instances outside the COCO60 evaluation subset, counted as false positives under AR@1 despite reflecting valid objectness.

Setting	AR Ovr	AR Sml	AR Med	AR Lrg
$\mathcal{L}_{\mathrm{CE}}$ + $\mathcal{L}_{\mathrm{L1}}$	20.8	5.2	29.6	39.7
$\mathcal{L}_{\mathrm{CE}}$ + $\mathcal{L}_{\mathrm{SF}}$	27.2	5.0	40.8	52.0
$\mathcal{L}_{\mathrm{obj}}$ + $\mathcal{L}_{\mathrm{L1}}$	18.5	0.7	22.3	50.0
$\mathcal{L}_{\mathrm{obj}}$ + $\mathcal{L}_{\mathrm{L1}}$ ( $\mathcal{N}_{k}$ )	33.4	12.9	48.1	46.8
$\mathcal{L}_{\mathrm{obj}}$ + $\mathcal{L}_{\mathrm{SF}}$	48.7	11.0	71.1	91.9
$\mathcal{L}_{\mathrm{ROS}}$ + $\mathcal{L}_{\mathrm{SF}}$	23.6	1.5	32.2	56.3

Table 6: Ablation of supervision design on the VOC

\rightarrow

COCO60 split.

\mathcal{L}_{\mathrm{CE}}

and

\mathcal{L}_{\mathrm{obj}}

denote conventional (foreground vs background) and proposed objectness supervision, respectively.

\mathcal{L}_{\mathrm{L1}}

\mathcal{L}_{\mathrm{L1}}(\mathcal{N}_{k})

, and

\mathcal{L}_{\mathrm{SF}}

denote global regression, local-neighborhood regression, and the proposed scale-field loss.

\mathcal{L}_{\mathrm{ROS}}

indicates

\mathcal{L}_{\mathrm{obj}}

without

\mathcal{L}_{\mathrm{ctr}}

term.

Figure 7: Effect of the percentile threshold $\tau_{\ell}$ on AR@1000 (IoU $\geq 0.5$ , VOC20 $\to$ COCO60 split). Left: size-wise recall (small, medium, large, overall)@1000. Right: recall at proposal budgets (AR@ ${1,10,100,1000}$ ). The dashed vertical line marks the chosen 60th percentile. Recall remains stable up to the 80th percentile and degrades sharply beyond 90th.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8: Objectness bias visualization on COCO val2017 . Green boxes denote labeled COCO objects, while red boxes indicate objects unlabeled during training. Filled circles show objectness scores, color-coded from low ( $\bullet$ ) to high ( $\bullet$ ). CenterNet [ 36 ] and FCOS [ 26 ] assign consistently low scores to unlabeled objects, revealing objectness bias caused by foreground-background supervision. In contrast, B-FOR assigns similar scores to both labeled and unlabeled objects, substantially reducing objectness bias.

Figure 9: From left to right: input image $I$ , object-center score overlays for large, medium, and small scales, and final detections $D(I)$ .

Figure 10: Qualitative top-30 detections on COCO images using a model trained on COCO80. Rows 1, 3, 5 are ground truth (GT); Rows 2, 4, 6 show the model predictions.

Figure 11: Failure cases: B-FOR (trained on VOC20) sometimes activates on salient object parts rather than complete instances, since box-level supervision does not distinguish objects from discriminative parts.

Protocol	Training strategy	AR Ovr	AR Sml	AR Med	AR Lrg
VOC 17 $\rightarrow$ VOC 3	Staged	61.5	43.1	70.2	60.0
VOC 17 $\rightarrow$ VOC 3	Joint	80.3	36.2	81.7	93.4
VOC 20 $\rightarrow$ COCO 60	Staged	34.7	11.4	51.6	56.6
VOC 20 $\rightarrow$ COCO 60	Joint	48.7	11.0	71.1	91.8
COCO 80 $\rightarrow$ OpenImages 490	Staged	30.8	7.9	43.8	30.2
COCO 80 $\rightarrow$ OpenImages 490	Joint	60.2	7.2	45.6	77.5

Appendix

Table 7: Comparison of staged and joint training across cross-dataset protocols. Staged training first optimizes object-center learning and then trains the scale-field branch, while joint training optimizes the full objective end-to-end. All settings use the same architecture and inference pipeline. We report AR@1000 at IoU

\geq 0.5

overall and by object size.

Figure 12: AR1/AR10/AR100/AR1000 vs. top- $K$ peaks retained per pyramid level, VOC20 $\to$ COCO60. All curves saturate by $K{\approx}600$ ; the inset zooms into $K\leq 100$ , where AR100/AR1000 are still rising.

Explore similar work

Aug 13, 2026

Towards Sparsely Annotated Open-World Object Detection

Real-world object detection operates under ambiguous supervision, where unlabeled regions may correspond to missing annotations of known objects or genuinely unknown categories. These challenges have been addressed separately in Sparsely Annotated Object Detection (SAOD) and Open-World Object Detection (OWOD). In practice, their co-occurrence remains an open problem. To address this problem, we introduce Sparsely Annotated Open-World Object Detection (SA-OWOD), a new task that jointly considers sparse supervision and the presence of unseen categories. We propose Dual-Perspective Object Discovery (DPOD), a unified framework that jointly models unlabeled known and unknown instances via two complementary mechanisms. The Known Target Recovery Module (KTRM) recovers supervision for unlabeled known instances and explicitly regularizes the feature space to separate known and unknown representations. Complementarily, the Dual-Disagreement Target Generator (DDTG) identifies reliable unknown candidates through cross-view semantic inconsistency. By integrating these modules, DPOD resolves contradictory supervision signals caused by ambiguous unlabeled regions. As a result, it prevents misclassification between known and unknown objects and stabilizes the decision boundaries. Experimental results on sparsely annotated open-world benchmarks demonstrate that the proposed method outperforms existing open-world detection methods, particularly in detecting unknown objects.

HeeJu Han, AJeong Kim, Jinsun Park

Pusan National University, Republic of Korea

Open-Vocabulary Object Detection Object Detection

Ask PDF

Aug 7, 2026

ECAD: Expanding Class-Agnostic Detection Beyond Thing-Centric Objectness

Object detection is a fundamental task in visual perception, providing structured region representations for recognition, grounding, reasoning, and interaction. However, existing detection paradigms largely inherit a thing-centric notion of objectness, where detectors are mainly trained to localize discrete and countable object instances. Consequently, many semantically meaningful visual elements, such as sky, road, grassland, water, and sports courts, are often absorbed into the background despite their importance for scene understanding and spatial reasoning. In this paper, we formulate Expanded Class-Agnostic Detection (ECAD), a new setting that aims to discover category-agnostic visual candidates beyond conventional thing-centric objects. To support this setting, we construct BTCO-Bench, a Beyond Thing-Centric Objectness benchmark with category-agnostic box annotations covering both real-world and cross-domain scenarios. We further propose ECADet, a lightweight DETR-based detector built upon a frozen DINOv3 encoder, and introduce Geometry-Aware Expert Regression (GAER) and Prototype-Guided Query Modulation (PGQM) to improve localization and objectness estimation for diverse visual elements, respectively. Extensive experiments show that ECADet consistently outperforms representative class-agnostic and proposal-based detectors on BTCO-Bench, demonstrating the effectiveness of expanded objectness discovery. Code and benchmark will be released.

Liang Wan, Zixin Ren, Yupeng Zhang +2

College of Intelligence and Computing, Tianjin University. · 2Key Research Center for Surface Monitoring and Analysis of Relics, State Administration of Cultural Heritage.

Object Detection

Ask PDF

May 6, 2026

Reference-based Category Discovery: Unsupervised Object Detection with Category Awareness

Traditional one-shot detection methods have addressed the closed-set problem in object detection, but the high cost of data annotation remains a critical challenge. General unsupervised methods generate pseudo boxes without category labels, thus failing to achieve category-aware classification. To overcome these limitations, we propose Reference-based Category Discovery (RefCD), an unsupervised detector that enables category-aware\footnotemark[1] detection without any manually annotated labels. It leverages feature similarity between predicted objects and unlabeled reference images. Unlike previous unsupervised methods that lack category guidance and one-shot methods which require labeled data, RefCD introduces a carefully designed feature similarity loss to explicitly guide the learning of potential category-specific features. Additionally, RefCD supports category-agnostic detection without reference images, serving as a unified framework. Comprehensive quantitative and qualitative analysis of category-aware and category-agnostic detection results demonstrates its effectiveness, and RefCD can learn category information in an unsupervised paradigm even without category labels.

Yichen Li, Qiankun Liu, Ying Fu

Department of Computer Science Beijing Institute of Technology · University of Science and Technology Beijing

Representation Learning Object Detection

Ask PDF