OCBI: Order-Sensitive Cross-Branch Interaction for Image Manipulation Localization
Abstract
Image manipulation localization combines semantic structure with forensic traces to identify modified pixels. Within-branch spatial mixing and RGB-guided cross-branch mixing can produce different messages when composed in opposite orders. We propose OCBI, an extension of Co-Transformers that models the discrepancy between these two composition orders for localization. A local cross-branch fusion module uses the commutator of RGB-guided cross-attention and forensic self-attention to add a learnable signed correction to feature transfer. Pixelwise adaptive multiscale aggregation then combines spatial contexts before prediction. OCBI uses the Co-Transformers backbone and supervision. We compare OCBI with retrained Co-Transformers using CASIA-only task training, a 50-epoch budget, and final-epoch evaluation. Across five clean test datasets, OCBI achieves a macro PixelF1 of 0.5773, exceeding the baseline by 0.43 percentage points. Under JPEG compression, Gaussian blur, and resizing on three datasets, its average PixelF1 over 21 dataset-transformation conditions exceeds the baseline by 0.95 percentage points.
Figures & tables
| Split | List entries | Role |
|---|---|---|
| CASIA v2.0 train | 4 865 | task training |
| CASIA v2.0 validation | 258 | monitoring |
| CASIA v1.0 | 920 | clean test |
| COVERAGE | 100 | clean test |
| Columbia | 180 | clean test |
| NIST16 subset | 240 | clean test |
| Setting | Configuration |
|---|---|
| Training data and input | CASIA-only training split; inputs; baseline mask construction and geometric/photometric augmentation |
| Objective | |
| Optimizer | AdamW with , , and |
| Learning rate and regularization | Shared parameters: , weight decay ; new OCBI parameters: , weight decay |
| Schedule | 2-epoch warm-up followed by cosine decay to |
| Hardware and batch size | Four NVIDIA A800-SXM4-80GB GPUs; per-device/global batch size ; no gradient accumulation |
| Dataset | Co-Transformers F1 | OCBI F1 | Co-Transformers AUC | OCBI AUC |
|---|---|---|---|---|
| CASIA v1.0 | 0.7409 | 0.7452 | 0.9536 | 0.9572 |
| COVERAGE | 0.2853 | 0.2994 | 0.8868 | 0.8826 |
| Columbia | 0.8591 | 0.8613 | 0.9288 | 0.9313 |
| NIST16 subset | 0.5605 | 0.5607 | 0.9072 | 0.8928 |
| IMD2020 | 0.4192 | 0.4197 | 0.8641 | 0.8724 |
| Macro average | 0.5730 | 0.5773 | 0.9081 | 0.9073 |
| Transformation | CASIA v1.0 | COVERAGE | Columbia | |||
|---|---|---|---|---|---|---|
| Baseline | OCBI | Baseline | OCBI | Baseline | OCBI | |
| JPEG Q90 | 0.7035 | 0.7073 | 0.1965 | 0.1996 | 0.8179 | 0.8402 |
| JPEG Q70 | 0.6126 | 0.5992 | 0.2055 | 0.1707 | 0.6692 | 0.7448 |
| JPEG Q50 | 0.4885 | 0.4746 | 0.1906 | 0.1422 | 0.5767 | 0.6709 |
| Blur | 0.6577 | 0.6694 | 0.1601 | 0.1264 | 0.7474 | 0.7681 |
| Blur | 0.5749 | 0.5767 | 0.1432 | 0.1768 | 0.5810 | 0.6089 |
Image Forgery Localization
Image forgery localization identifies manipulated or AI-generated regions in images, often alongside image-level authenticity detection.
See all 84 papers