TSDCRF: Balancing Privacy and Multi-Object Tracking via Time-Series CRF and Normalized Control Penalty
Authors: Bo Ma, Jinsong Wu, Weiqi Yan
Organizations: School of Mathematics and Computer Engineering, Auckland University of Technology,Auckland 1024, New Zealand · Department of Electrical Engineering, University of Chile, Santiago, Chile
Multi-object tracking in video often requires appearance or location cues that can reveal sensitive identity information, while adding privacy-preserving noise typically disrupts cross-frame association and causes ID switches or target loss. We propose TSDCRF, a plug-in refinement framework that balances privacy and tracking by combining three components: (i) (ε,δ)-differential privacy via calibrated Gaussian noise on sensitive regions under a configurable privacy budget; (ii) a Normalized Control Penalty (NCP) that down-weights unstable or conflicting class predictions before noise injection to stabilize association; and (iii) a time-series dynamic conditional random field (DCRF) that enforces temporal consistency and corrects trajectory deviation after noise, mitigating ID switches and resilience to trajectory hijacking. The pipeline is agnostic to the choice of detector and tracker (e.g., YOLOv4 and DeepSORT). We evaluate on MOT16, MOT17, Cityscapes, and KITTI. Results show that TSDCRF achieves a better privacy--utility trade-off than white noise and prior methods (NTPD, PPDTSA): lower KL-divergence shift, lower tracking RMSE, and improved robustness under trajectory hijacking while preserving privacy. Source code in https://github.com/mabo1215/TSDCRF.git
Figures & tables
Fig. 1: Segmentation and tracking results. Top: three consecutive frames from MOT16. Bottom: segmentation and trajectory after target identification.
Fig. 2: Tracking Result For Vehicle
Fig. 3: Flowchart of the proposed framework: detector → tracker → privacy module → TSDCRF refinement.
Fig. 4: Motion trajectory prediction with the CRF (Fig. 4). The time-series CRF enforces temporal consistency and reduces position deviation after privacy noise.
Fig. 5: The performance of the proposed methods. (a) Defense under trajectory hijacking attack; (b) defense under blind target object detection attack.
Fig. 6: Comparison of sensitive area distributions
Fig. 7: Comparison of degree distribution and related parameter k .
Fig. 8: Comparison of degree distribution and sampling steps
Fig. 9: Privacy retrieval vs. path/structure: comparison of different methods (see text for metric).
Fig. 10: Privacy retrieval frequency vs. privacy budget ϵ .
Fig. 11: Privacy appearance frequency vs. parameter k .
Fig. 12: Privacy appearance frequency vs. sampling step.
Fig. 13: Root-mean-square deviation of tracking bounding box
Method
ε=0.5
ε=1.0
ε=1.5
NCP+Gaussian (Ours)
0.18
0.21
0.24
PPDTSA
0.32
0.41
0.48
NTPD
0.52
0.61
0.68
White noise
2.18
2.52
2.78
TABLE I: RMSE of tracking bounding box (from Fig. 13 ), averaged over MOT16, MOT17, Cityscapes, and KITTI. Lower RMSE = better tracking accuracy.
Method
ε=0.25
ε=0.5
ε=0.75
ε=1.0
NCP+Gaussian (Ours)
21.2
24.1
26.3
28.0
White noise
17.8
19.5
20.9
22.1
NTPD
19.6
22.4
24.5
26.2
PPDTSA
20.1
23.0
25.0
26.8
TABLE II: PSNR (dB) of perturbed vs. original bounding-box coordinates (sensitive detections), averaged over MOT16, MOT17, Cityscapes, and KITTI. Higher PSNR = less distortion. Computed from MSE via PSNR=10log10(MAX2/MSE) .
Dataset
ε=0.25
ε=0.5
ε=0.75
ε=1.0
MOT16
21.5
24.3
26.5
28.2
MOT17
21.0
23.9
26.1
27.8
Cityscapes
20.8
23.8
26.0
27.7
KITTI
21.4
24.2
26.2
27.9
TABLE III: PSNR (dB) for NCP+Gaussian (Ours) per dataset at different ε . Explains dataset-wise differences seen in Figs. 7 , 6 , and 13 .
Dashcam videos collected by autonomous or assisted-driving systems are increasingly shared for safety auditing and model improvement. Even when explicit GPS metadata are removed, an attacker can still infer the recording location by matching background visual cues (e.g., buildings and road layouts) against large-scale street-view imagery. This paper studies location-privacy leakage under a background-based retrieval attacker, and proposes PPEDCRF, a privacy-preserving enhanced dynamic conditional random field framework that injects calibrated perturbations only into inferred location-sensitive background regions while preserving foreground detection utility. PPEDCRF consists of three components: (i) a dynamic CRF that enforces temporal consistency to discover and track location sensitive regions across frames, (ii) a normalized control penalty (NCP) that allocates perturbation strength according to a hierarchical sensitivity model, and (iii) a utility-preserving noise injection module that minimizes interference to object detection and segmentation. Experiments on public driving datasets demonstrate that PPEDCRF significantly reduces location-retrieval attack success (e.g., Top-k retrieval accuracy) while maintaining competitive detection performance (e.g., mAP and segmentation metrics) compared with common baselines such as global noise, white-noise masking, and feature-based anonymization. The source code is in https://github.com/mabo1215/PPEDCRF.git
Bo Ma, Jinsong Wu, Weiqi Yan +2
Auckland University of Technology, New Zealand. · the Department of Electrical Engineering, Universidad de Chile, Chile.
We propose PPEDCRF, a calibrated selective perturbation framework that protects background-based location privacy in released video frames against gallery-based retrieval attackers. Even after GPS metadata are stripped, an adversary can geolocate a frame by matching its background visual cues to geo-tagged reference imagery; PPEDCRF mitigates this threat by estimating location-sensitive background regions with a dynamic conditional random field (DCRF), rescaling perturbation strength with a normalized control penalty (NCP), and injecting Gaussian noise only inside the inferred regions via a DP-style calibration rule. On a controlled paired-scene retrieval benchmark with eight attacker backbones and three noise seeds, PPEDCRF reduces ResNet18 Top-1 retrieval accuracy from 0.667 to 0.361±0.127 at σ0=8 while preserving 36.14dB PSNR -- an ≈6dB quality advantage over global Gaussian noise. Transfer across the eight-backbone seed-averaged benchmark is broadly supportive (23 of 24 backbone-gallery cells show negative Δ), while appendix-scale confirmation identifies MixVPR as a remaining adverse-transfer exception. Matched-operating-point analysis shows that PPEDCRF and global Gaussian noise converge in Top-1 privacy at equal utility, so the practical benefit is spatially concentrated perturbation that preserves higher visual quality at any given noise scale rather than stronger matched-utility privacy. Code: https://github.com/mabo1215/PPEDCRF
Bo Ma, Weiqi Yan, Jinsong Wu
Resideo Technologies Inc., Scottsdale, Arizona, USA · Auckland University of Technology, Auckland, New Zealand · Guilin University of Electronic Technology, Guilin, China
Temporal information is crucial for visual tracking, but existing multi-frame trackers are vulnerable to model drift caused by naively aggregating noisy historical predictions. In this paper, we introduce DTPTrack, a lightweight and generalizable module designed to be seamlessly integrated into existing trackers to suppress drift. Our framework consists of two core components: (1) a Temporal Reliability Calibrator (TRC) mechanism that learns to assign a per-frame reliability score to historical states, filtering out noise while anchoring on the ground-truth template; and (2) a Temporal Guidance Synthesizer (TGS) module that synthesizes this calibrated history into a compact set of dynamic temporal priors to provide predictive guidance. To demonstrate its versatility, we integrate DTPTrack into three diverse tracking architectures--OSTrack, ODTrack, and LoRAT-and show consistent, significant performance gains across all baselines. Our best-performing model, built upon an extended LoRATv2 backbone, sets a new state-of-the-art on several benchmarks, achieving a 77.5% Success rate on LaSOT and an 80.3% AO on GOT-10k.
Yuqing Huang, Liting Lin, Weijun Zhuang +2
Harbin Institute of Technology, Shenzhen · Pengcheng Laboratory · Lero, the Research Ireland Centre for Software, University of Limerick +1