PriCBMs: Multi-granular Concept Bottleneck Vision-Language Models for Visual Privacy Recognition
Organizations: School of Intelligence Science and Engineering, Qinghai Minzu University, Xining 810007, China · Tianjin Key Laboratory of Interventional Brain-Computer Interface and Intelligent Rehabilitation and Academy for Advanced Interdisciplinary Studies, Nankai University, Tianjin 300350, China · School of Computing and Information Systems, Singapore Management University, Singapore 178902, Singapore
Abstract
Widespread image sharing on social media has increased the need for visual privacy recognition to identify inadvertently exposed privacy information. However, multi-label visual privacy recognition presents two challenges. First, privacy categories represent high-level semantic attributes and do not have a one-to-one correspondence with visual objects. Different visual objects may involve the same privacy category, while the same type of object may contain different types of privacy-sensitive information. In this setting, direct prediction methods generally lack explicit guidance for identifying category-relevant privacy cues, leaving this process implicit in the learned image-to-label mapping and potentially limiting classification performance. Second, privacy cues are often distributed across image regions and may not be adequately represented by whole-image features. To address these challenges, we propose Privacy Concept Bottleneck Models (PriCBMs). By introducing privacy-related concepts as intermediate semantic representations, PriCBMs first identify category-relevant privacy cues through visual-concept matching and then predict privacy categories based on concept responses. This process makes privacy predictions more interpretable. To improve the quality of these concepts, we design a concept selection strategy based on target-category relevance, cross-category discriminability, and semantic diversity. Meanwhile, we introduce a global-region modeling module to capture localized privacy evidence while retaining global context. Experiments on VISPR and FineViP show that PriCBMs outperform evaluated competitors across all four multi-label metrics. Ablation studies and concept-quality analyses support the effectiveness of the proposed design. Code is available at https://github.com/sun2293674491-dotcom/PriCBMs.
Figures & tables
| Backbone | Method | MAP-c | MAP-s | Mic-F1 | Mac-F1 |
|---|---|---|---|---|---|
| ResNet50 | BCE | 49.54 | 85.36 | 78.20 | 34.80 |
| Focal | 48.84 | 84.71 | 77.39 | 35.96 | |
| TWL | 50.02 | 81.08 | 66.34 | 38.24 | |
| MMBipolar | 53.51 | 84.39 | 78.96 | 36.13 | |
| BipolarBCE | 51.39 | 84.86 | 78.59 | 38.91 | |
| BipolarFocal | 50.45 | 83.86 | 77.77 | 39.90 |
| Backbone | Method | MAP-c | MAP-s | Mic-F1 | Mac-F1 |
|---|---|---|---|---|---|
| ResNet50 | BCE | 67.46 | 80.69 | 68.40 | 62.03 |
| Focal | 66.90 | 80.26 | 67.66 | 61.71 | |
| TWL | 66.96 | 81.08 | 60.41 | 57.29 | |
| MMBipolar | 65.11 | 79.22 | 66.54 | 59.98 | |
| BipolarBCE | 67.38 | 81.07 | 67.91 | 62.34 | |
| BipolarFocal | 67.70 | 81.20 | 68.11 | 62.01 |
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Concept Bottleneck Models
Concept Bottleneck Models route predictions through human-understandable concepts that people can inspect, correct, or intervene on.
See all 92 papers