BEM: Training-Free Background Embedding Memory for False-Positive Suppression in Real-Time Fixed-Background Camera
Authors: Junwoo Park, Jangho Lee, Sunho Lim
Organizations: Department of Computer Science and Engineering, Incheon National University, Incheon, Republic of Korea · Army Artificial Intelligence Center, Republic of Korea Army, Gyeryong, Republic of Korea · Department of Computer Science, Texas Tech University, Lubbock, TX, USA
Pretrained detectors perform well on benchmarks but often suffer performance degradation in real-world deployments due to distribution gaps between training data and target environments. COCO-like benchmarks emphasize category diversity rather than instance density, causing detectors trained under per-class sparsity to struggle in dense, single- or few-class scenes such as surveillance and traffic monitoring. In fixed-camera environments, the quasi-static background provides a stable, label-free prior that can be exploited at inference to suppress spurious detections. To address the issue, we propose Background Embedding Memory (BEM), a lightweight, training-free, weight-frozen module that can be attached to pretrained detectors during inference. BEM estimates clean background embeddings, maintains a prototype memory, and re-scores detection logits with an inverse-similarity, rank-weighted penalty, effectively reducing false positives while maintaining recall. Empirically, background-frame cosine similarity correlates negatively with object count and positively with Precision-Confidence AUC (P-AUC), motivating its use as a training-free control signal. Across YOLO and RT-DETR families on LLVIP and simulated surveillance streams, BEM consistently reduces false positives while preserving real-time performance. Our code is available at https://github.com/Leo-Park1214/Background-Embedding-Memory.git
Figures & tables
Figure 1: Persons-per-image distributions comparing COCO/VOC with LLVIP. LLVIP contain substantially denser per-class instances, reflecting real surveillance and traffic scenes [ 8 ] .
Figure 2: Relationship between background–frame cosine similarity c and (a) number of objects, and (b) P-AUC on LLVIP.
Figure 3: Overview of the Background Embedding Memory (BEM) pipeline. During inference, BEM attaches to a frozen detector without retraining. The module first extracts a background embedding from fixed-scene frames via the shared backbone, computes cosine similarity with the current input embedding, and re-scores the detector’s output logits to suppress background-induced false positives.
Table 1: Quantitative comparison of [email protected] and P-AUC across six detectors on the LLVIP dataset. Base denotes the baseline detector without BEM, while BEM applies the proposed similarity-driven re-scoring.
Figure 5
Figure 5: Ablation study of the BEM penalty scale α and temperature γ across different detector backbones. Each curve reports [email protected] as a function of α for several fixed values of γ .
Symbol
Range
Role
Description
α
[0.0,1.0]
Penalty scale
Controls suppression strength in logit space
γ
(0.0,1.0]
Temperature
Normalizes penalty magnitude (sharpness)
L
-
Temporal window
Number of frames for background averaging
K
-
Prototype count
Number of frames aggregated for one prototype
Table 3: Summary of BEM hyperparameters. L denotes the number of recent frames used for temporal background estimation, while K denotes the number of background embeddings aggregated to form a single prototype. In our experiments, we set K=L .
Figure 6: Qualitative comparison of detection results. The top two rows show results from YOLO-based detectors, while the bottom two rows correspond to DETR-based detectors. (a) Ground-truth bounding boxes, (b) Detection results of the baseline detector, (c) Baseline detections with false positives highlighted with green color, and (d) Detection results with the proposed BEM applied.
Object detectors are typically trained under closed-set supervision, where unlabeled regions are implicitly treated as background. Under incomplete annotations, this assumption introduces objectness bias: visually valid but unlabeled objects are used as negatives, tying objectness to the annotated taxonomy rather than generic object structure. This limitation is particularly problematic for class-agnostic and open-world detection. This paper proposes Background-Free Objectness Learning (B-FOR), a dense class-agnostic detection framework that learns objectness without explicit background supervision on unlabeled regions. B-FOR formulates detection as the prediction of dense multi-scale object-center and scale fields, from which object hypotheses emerge as local spatial structures. Supervision is confined to reliable annotated regions through spatially structured soft targets, avoiding foreground-background discrimination. To support decoding from emergent local maxima, the paper further introduces displacement-aware scale fields that model object extent as a spatially varying property of the learned objectness field. Experiments on PASCAL VOC, MS-COCO, and Open Images demonstrate strong generalization to unseen categories and cross-dataset object distributions. B-FOR improves recall by more than +10 AR points over prior class-agnostic baselines. Ablation studies show that both localized objectness supervision and displacement-aware scale fields are critical for class-agnostic localization under incomplete annotations. Code available at: https://github.com/Daniaawan/B-FOR.
Dania Batool, Liliana Lo Presti, Marco La Cascia +1
Department of Engineering University of Palermo Palermo, Italy · ICAR National Research Council of Italy Palermo, Italy
Transformer-based detectors have advanced small-object detection, but they often remain inefficient and vulnerable to background-induced query noise, which motivates deep decoders to refine low-quality queries. We present HELP (Heatmap-guided Embedding Learning Paradigm), a noise-aware positional-semantic fusion framework that studies where to embed positional information by selectively preserving positional encodings in foreground-salient regions while suppressing background clutter. Within HELP, we introduce Heatmap-guided Positional Embedding (HPE) as the core embedding mechanism and visualize it with a heatbar for interpretable diagnosis and fine-tuning. HPE is integrated into both the encoder and decoder: it guides noise-suppressed feature encoding by injecting heatmap-aware positional encoding, and it enables high-quality query retrieval by filtering background-dominant embeddings via a gradient-based mask filter before decoding. To address feature sparsity in complex small targets, we integrate Linear-Snake Convolution to enrich retrieval-relevant representations. The gradient-based heatmap supervision is used during training only, incurring no additional gradient computation at inference. As a result, our design reduces decoder layers from eight to three and achieves a 59.4% parameter reduction (66.3M vs. 163M) while maintaining consistent accuracy gains under a reduced compute budget across benchmarks. Code Repository: https://github.com/yidimopozhibai/Noise-Suppressed-Query-Retrieval
Yangchen Zeng, Zhenyu Yu, Dongming Jiang +5
Southeast University · Fudan University · The University of Texas at Dallas +6
Generic parameter-efficient fine-tuning (PEFT) methods transferred from language models can fail silently on real-time detectors, whose heterogeneous operators and detection-specific components impose placement constraints absent from regular Transformer stacks. We propose YOLO-PEFT, a structure-aware framework that formulates adapter placement as an auditable constraint-planning problem. Given a detector graph, a PEFT request, and a resource budget, YOLO-PEFT assigns operator and semantic roles, evaluates explicit operator-validity, detector-semantic, graph-interface, and deployment predicates, records a reason code for each excluded module, and either emits a budgeted target-module plan or returns Refuse before training. Under the official VOC07+12 trainval-to-VOC07 test protocol, planner-selected RS-LoRA reaches 0.7138 and 0.7307 mAP50-95 on YOLO11s and YOLO12s, respectively, compared with 0.6428 and 0.6662 for Full-SFT. On RT-DETR-L, all seven evaluated LoRA-family configurations cross the predefined catastrophic threshold, supporting a calibrated Refuse-to-Full-SFT decision within the evaluated coverage. A controlled YOLO11 audit further shows that LoRA reduces peak training memory by 43.9 percent, although training takes 1.72 times longer. Within the evaluated detector families, placement policies, and calibration coverage, YOLO-PEFT replaces manual target-module trial and error with explicit, inspectable planning while preserving verified train-save-merge-export paths; refusal on unseen detector architectures remains an open validation problem. Project Page: github.com/Tencent/YOLO-Master