Consistency Beyond Contrast: Enhancing Open-Vocabulary Object Detection Robustness via Contextual Consistency Learning

Bozhao Li; Shaocong Wu; Tong Shao; Senqiao Yang; Qiben Shan; Zhuotao Tian; Jingyong Su

cs.CVMar 27, 2026

Consistency Beyond Contrast: Enhancing Open-Vocabulary Object Detection Robustness via Contextual Consistency Learning

Authors: Bozhao Li, Shaocong Wu, Tong Shao, Senqiao Yang, Qiben Shan, Zhuotao Tian, Jingyong Su

Organizations: Harbin Institute of Technology, Shenzhen · Pengcheng Laboratory · CUHK

Abstract

Recent advances in open-vocabulary object detection focus primarily on two aspects: scaling up datasets and leveraging contrastive learning to align language and vision modalities. However, these approaches often neglect internal consistency within a single modality, particularly when background or environmental changes occur. This lack of consistency leads to a performance drop because the model struggles to detect the same object in different scenes, which reveals a robustness gap. To address this issue, we introduce Contextual Consistency Learning (CCL), a novel framework that integrates two key strategies: Contextual Bootstrapped Data Generation (CBDG) and Contextual Consistency Loss (CCLoss). CBDG functions as a data generation mechanism, producing images that contain the same objects across diverse backgrounds. This is essential because existing datasets alone do not support our CCL framework. The CCLoss further enforces the invariance of object features despite environmental changes, thereby improving the model's robustness in different scenes. These strategies collectively form a unified framework for ensuring contextual consistency within the same modality. Our method achieves state-of-the-art performance, surpassing previous approaches by +16.3 AP on OmniLabel and +14.9 AP on D3. These results demonstrate the importance of enforcing intra-modal consistency, significantly enhancing model generalization in diverse environments. Our code is publicly available at: https://github.com/bozhao-li/CCL.

Figures & tables

Figure 1: Performance and robustness comparison of different methods. (a) Our approach, with Contextual Consistency Learning, achieves the best overall results, reaching a normalized score of 1 in all metrics. (b,c) Benchmark backgrounds are altered to test robustness. Tested on $D^{3}$ BC , baseline methods degrade, while ours remains stable. See Section 4.3 for details.

Figure 2: Overview of our approach. CBDG generates

D_{j}

via Categorical Augmentation, Background Generation and Background Replacement. CCL training uses

D_{j}

with CCLoss added to total loss.

Figure 3: CBDG Pipeline. We use ChatGPT to generate background prompts for a diffusion model, enabling diverse background synthesis. For single-class images, CBDG augments object categories before background replacement. For multi-class images, CBDG replaces only the background.

Figure 4: Four groups of images are shown, each composed of four sub-images: the leftmost sub-image in every group is the original, while the remaining three display background replacements.

Figure 5: CCL Framework. Visual and textual features are encoded, with regional features pooled into CAAF. Consistency loss is applied within each modality.

	OmniLabel							$D^{3}$
Method	AP	AP-c	AP-d	AP-dP	AP-dS	AP-dM	AP-dL	FULL	PRES	ABS
Detic [ 65 ]	8.0	15.6	5.4	8.0	5.7	5.4	6.2	-	-	-
OFA-DOD [ 52 ]	-	-	-	-	-	-	-	21.6	23.7	15.4
RelationLLM-L [ 51 ]	-	-	-	-	-	-	-	24.3	24.6	23.4
GN-GLIP [ 64 ]	22.2	27.2	18.8	29.0	-	-	-	21.4	20.6	23.7
GN-FIBER [ 64 ]	28.1	32.1	25.1	36.5	-	-	-	26.0	25.2	28.1

Table 1: Performance of our method compared with SOTA methods.

Figure 6: Examples of $D^{3}_{\mathrm{C}}$ Perturbations.

	$D^{3}$ BC			$D^{3}_{\mathrm{C}}$
Method	FULL	PRES	ABS	FULL	mFULL	rFULL
GLIP-T	16.8	15.8	20.4	19.1	13.6	71.2
+ours	29.6	28.9	31.9	30.0	21.7	72.3
FIBER-B	20.1	18.7	24.7	22.7	16.7	73.6
+ours	33.1	32.8	34.0	37.6	27.5	73.1

Table 2: Performance comparison on

D^{3}

BC and

D^{3}_{\mathrm{C}}

	OmniLabel			$D^{3}$
Method	AP	AP-c	AP-d	FULL	PRES	ABS
GLIP-T	19.3	23.6	16.4	19.1	18.3	21.5
+data	24.8	29.2	21.8	23.2	22.5	25.3
+ours	32.2	36.1	28.8	30.0	29.2	32.3
FIBER-B	25.7	30.3	22.3	22.7	21.5	26.0
+data	32.7	35.8	29.6	29.1	28.3	31.2

Table 3: Ablation study on the effects of Contextual Bootstrapped Data Generation (CBDG) and Contextual Consistency Loss (CCLoss). “+data” denotes using CBDG without CCLoss, “+ours” combines both components.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Tuning	$\lambda_{I}$	$\lambda_{T}$	OmniLabel (AP)	$D^{3}$ (FULL)
0	0	0	32.7	29.2
1	0.5	0.5	35.8	31.5
2	0.2	0.2	38.2	34.0
3	0.1	0.1	39.9	36.1
4	0.15	0.15	41.2	37.1
5	0.15	0.1	41.7	37.4

Appendix

Table 4: Tuning on the FIBER baseline

Method	AP	Ap-c	Ap-d	FULL	PRES	ABS
FIBER-B+ours	42.0	44.1	39.2	37.6	37.2	38.8
FIBER-B + BBox copy-paste	23.3	27.0	20.6	20.9	20.4	22.6

Appendix

Table 5: Comparison Between Our Method and Simple BBox Copy-Paste Based on the FIBER Baseline

Method	FULL	PRES	ABS
GLIP-T+ours	29.7	28.9	32.1
FIBER-B+ours	37.5	37.1	38.6

Appendix

Table 6: Results on 2,304

D^{3}

Images from Cityscapes and Indoor Urban Environments

Model	Val			Test
Model	R@1	R@5	R@10	R@1	R@5	R@10
GLIP-T	85.7	95.4	96.9	85.7	95.8	97.2
GLIP-T+ours	88.5	95.9	97.2	88.8	96.2	97.2
FIBER-B	87.1	96.1	97.4	87.4	96.4	97.6
FIBER-B+ours	89.9	96.6	97.7	90.0	96.8	97.7

Appendix

Table 7: Recall@K metrics for both GLIP-T and FIBER-B before and after applying our method

Method(Baseline)	OmniLabel			$D^{3}$
Method(Baseline)	AP	AP-c	AP-d	FULL	PRES	ABS
IAM(FIBER-B)	20.5	24.6	18.0	18.4	17.8	20.3
+ours(FIBER-B)	26.1	31.2	22.2	23.5	22.8	26.1
CBDG+ours(FIBER-B)	32.4	34.6	29.4	30.3	30.0	31.4

Appendix

Table 8: Experiments on different generation method.

Method(Baseline)	OmniLabel			$D^{3}$
Method(Baseline)	AP	AP-c	AP-d	FULL	PRES	ABS
Matrix(FIBER-B)	29.8	32.4	27.5	27.9	26.8	30.0
Variance(FIBER-B)	30.2	33.2	28.6	28.8	28.1	30.5
CCLoss(FIBER-B)	32.4	34.6	29.4	30.3	30.0	31.4

Appendix

Table 9: Experiments on different consistency loss.

Method(Baseline)	OmniLabel			$D^{3}$
Method(Baseline)	AP	AP-c	AP-d	FULL	PRES	ABS
GLIP-T	19.3	23.6	16.4	19.1	18.3	21.5
+ours( $K_{r}$ =0.6)	27.2	30.4	24.7	26.2	25.2	29.3
+ours( $K_{r}$ =0.8)	30.1	34.1	27.4	28.9	28.1	31.3
FIBER-B	25.7	30.3	22.3	22.7	21.5	26.0
+ours( $K_{r}$ =0.6)	35.6	38.3	34.0	32.4	32.0	33.6

Appendix

Table 10: Impact of dataset size of the CCL approach, where

K_{r}

is the dataset reduction scaling factor.

Figure 7: Three types of background images we generate. Given a prompt from LLM, we use stable diffusion to generate random background images.

Figure 8: Failure cases of background replacement using inpainting method [ 57 ] . Each case consists of paired images: the left image shows the original input, while the right image demonstrates the replaced background with corresponding artifacts. Cases are presented in a two-per-row layout for comparison.

Open-Vocabulary Object Detection

Open-vocabulary object detection lets models locate and label objects using text categories beyond their fixed training labels.

See all 89 papers

Exploring Hierarchical Consistency and Unbiased Objectness for Open-Vocabulary Object Detection
Apr 25, 2026Sanghoon Lee, Geon Lee, Hyekang Park +1Open-Vocabulary Object Detection Probability Calibration
Yonsei University · Korea Institute of Science and Technology (KIST)
Parameter-Efficient Semantic Augmentation for Enhancing Open-Vocabulary Object Detection
Apr 6, 2026Weihao Cao, Runqi Wang, Xiaoyue Duan +3Open-Vocabulary Object Detection Vision Foundation Model Adaptation
State Key Laboratory of Advanced Rail Autonomous Operation, Beijing Jiaotong University · Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence, Beijing Jiaotong University · School of Computer Science & Technology, Beijing Jiaotong University +1
DeCo-DETR: Decoupled Cognition DETR for efficient Open-Vocabulary Object Detection
Apr 3, 2026Siheng Wang, Yanshu Li, Bohan Hu +12Open-Vocabulary Object Detection Detection Transformer
Jiangsu University · MBZUAI · Brown University +5

Apr 25, 2026J/K move · Enter open · S save

Exploring Hierarchical Consistency and Unbiased Objectness for Open-Vocabulary Object Detection

Sanghoon Lee, Geon Lee, Hyekang Park, Bumsub Ham

Yonsei University · Korea Institute of Science and Technology (KIST)

Open-Vocabulary Object Detection Probability Calibration VLM Adaptation Pseudo-Labeling Object Detection

Open paper AskPDF arXiv

Conventional object detectors typically operate under a closed-set assumption, limiting recognition to a predefined set of base classes seen during training. Open-vocabulary object detection (OVD) addresses this limitation by leveraging vision-language models (VLMs) to generate pseudo labels for novel object classes. However, existing OVD methods suffer from two critical drawbacks: (1) inaccurate class label assignments, as VLMs are optimized for image-level predictions rather than the region-level predictions required for pseudo labeling, and (2) unreliable objectness scores from region proposal networks (RPNs) trained exclusively on base object classes. To address these issues, we propose a novel pseudo labeling framework for OVD. Our approach introduces a hierarchical confidence calibration (HCC) technique, which ensures reliable class label estimation by assessing consistency across hierarchical semantic levels (class, super- and sub-category). We also present LoCLIP, a parameter-efficient adaptation of CLIP that incorporates an objectness token to mitigate base class bias problem of RPNs and provide reliable objectness estimations for novel object classes. Extensive experiments on standard OVD benchmarks, including COCO and LVIS, demonstrate that our approach clearly sets a new state of the art, validating the effectiveness of our approach. Project site: https://cvlab.yonsei.ac.kr/projects/HCC

Abstract

Open-Vocabulary Object Detection

Explore similar work

Exploring Hierarchical Consistency and Unbiased Objectness for Open-Vocabulary Object Detection

Parameter-Efficient Semantic Augmentation for Enhancing Open-Vocabulary Object Detection

DeCo-DETR: Decoupled Cognition DETR for efficient Open-Vocabulary Object Detection