Learning to Falsify: Hierarchical Multiple Instance Learning for High Resolution AI-Generated Image Detection
Organizations: Zhejiang Gongshang University, Hangzhou, Zhejiang, China · Zhejiang University, Hangzhou, Zhejiang, China · The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, Guangdong, China · Zhejiang Key Laboratory of Big Data and Future E-Commerce Technology, Hangzhou, Zhejiang, China
Abstract
Recent advances in generative models have made synthetic images increasingly photorealistic, posing growing challenges to reliable and generalizable detection of AI-generated images. Existing methods typically rely on holistic image-level classification, which may overlook local artifacts, especially in high-resolution images. To address this limitation, we formulate this task as a falsification-driven multiple instance learning problem, where a single fake patch can falsify the whole image, while a real image requires all patches to be authentic. Based on this formulation, we propose Learning to Falsify (LTF), a hierarchical multiple instance learning framework that models an image as multiple bags of instances across different spatial levels and aggregates their predictions under the falsification-inspired principle. We further introduce Cross-Shuffle during training to encourage the model to learn local artifact evidence rather than image-level shortcuts. Furthermore, we propose the Falsification-driven Test-time Aggregation Strategy (FATAS), a plug-and-play module that extends the same falsification-inspired principle to existing detectors by aggregating patch-level predictions. Extensive experiments demonstrate that LTF achieves strong generalization and robustness across multiple benchmarks, while FATAS substantially improves representative detectors in high-resolution scenarios without retraining. Code is available at https://github.com/HuiGuanLab/LTF.git
Figures & tables
| Method | Venue | Midjourney | SDv1.4 | SDv1.5 | ADM | GLIDE | Wukong | VQDM | BigGAN | mAcc | std |
| ResNet-50 ( He et al., 2016 ) | CVPR2016 | 54.9 | 99.9 | 99.7 | 53.5 | 61.9 | 98.2 | 56.6 | 52.0 | 72.1 | 22.6 |
| DeiT-S ( Touvron et al., 2021 ) | ICML2021 | 55.6 | 99.9 | 99.8 | 49.8 | 58.1 | 98.9 | 56.9 | 53.5 | 71.6 | 23.2 |
| Swin-T ( Liu et al., 2021 ) | ICCV2021 | 62.1 | 99.9 | 99.8 | 49.8 | 67.6 | 99.1 | 62.3 | 57.6 | 74.8 | 21.1 |
| CNNSpot ( Wang et al., 2020 ) | CVPR2020 | 52.8 | 96.3 | 95.9 | 50.1 | 39.8 | 78.6 | 53.4 | 46.8 | 64.2 | 22.6 |
| Spec ( Zhang et al., 2019 ) | WIFS2019 | 52.0 | 99.4 | 99.2 | 49.7 | 49.8 | 94.8 | 55.6 | 49.8 | 68.8 | 24.1 |
| F3Net ( Qian et al., 2020 ) | ECCV2020 | 50.1 | 99.9 | 99.9 | 49.9 | 50.0 | 99.9 | 49.9 | 49.9 | 68.7 | 25.8 |
| Method | SD Variants | Turbo Variants | LCM Variants | ControlNet Variants | DR Variants | mAcc | std | |||||||||||
| LDM | SDv1.4 | SDv1.5 | SDv2 | SDXL | SDXL- Refiner | SD- Turbo | SDXL- Turbo | LCM- SDv1.5 | LCM- SDXL | SDv1- Ctrl | SDv2- Ctrl | SDXL- Ctrl | SDv1- DR | SDv2- DR | SDXL- DR | |||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| CNNSpot ( Wang et al., 2020 ) | 99.87 | 99.91 | 99.90 | 97.55 | 66.25 | 86.55 | 86.15 | 72.42 | 98.26 | 61.72 | 97.96 | 85.89 | 82.84 | 60.93 | 51.41 | 50.28 | 81.12 | 17.6 |
| F3Net ( Qian et al., 2020 ) | 99.85 | 99.78 | 99.79 | 88.66 | 55.85 | 87.37 | 68.29 | 63.66 | 97.39 | 54.98 | 97.98 | 72.39 | 81.99 | 65.42 | 50.39 | 50.27 | 77.13 | 18.1 |
| CLIP/RN50 ( Radford et al., 2021 ) | 99.00 | 99.99 | 99.96 | 94.61 | 62.08 | 91.43 | 83.57 | 64.40 | 98.97 | 57.43 | 99.74 | 80.69 | 82.03 | 65.83 | 50.67 | 50.47 | 80.05 | 18.3 |
| GramNet ( Liu et al., 2020 ) | 99.40 | 99.01 | 98.84 | 95.30 | 62.63 | 80.68 | 71.19 | 69.32 | 93.05 | 57.02 | 89.97 | 75.55 | 82.68 | 51.23 | 50.01 | 50.08 | 76.62 | 17.0 |
| De-fake ( Sha et al., 2023 ) | 92.10 | 99.53 | 99.51 | 89.65 | 64.02 | 69.24 | 92.00 | 93.93 | 99.13 | 70.89 | 58.98 | 62.34 | 66.66 | 50.12 | 50.16 | 50.00 | 75.52 | 18.4 |
| Resolution Range | 0–900 | 900–1200 | 1200–1500 | 1500–2000 | 2000–2500 | 2500–3000 | 3000–5000 | Avg. | |
| CNNSpot | 58.46 | 56.37 | 58.73 | 63.14 | 67.42 | 60.90 | 66.30 | 58.90 | 61.28 |
| FreDetect | 63.67 | 55.63 | 57.31 | 58.90 | 58.16 | 67.78 | 57.23 | 51.06 | 58.72 |
| UnivFD | 67.00 | 58.35 | 62.20 | 65.95 | 66.05 | 58.20 | 58.15 | 60.05 | 61.99 |
| DIRE | 59.11 | 64.25 | 62.10 | 66.66 | 75.84 | 63.40 | 69.31 | 62.18 | 65.36 |
| AIDE | 65.87 | 57.29 | 49.90 | 58.23 | 51.88 | 65.31 | 61.04 | 42.16 | 56.46 |
| DRCT(ConvB) | 65.30 | 66.19 | 68.78 | 68.85 | 68.78 | 75.79 | 69.65 | 54.03 | 67.17 |
| Model | mAcc | Real Acc | Fake Acc | AUC | AP |
| Baseline | 91.86 | 97.17 | 86.55 | 98.22 | 98.17 |
| + MIL Head | 87.93 | 99.89 | 75.97 | 99.51 | 99.48 |
| + Cross Shuffle | 96.41 | 98.77 | 94.06 | 99.63 | 99.62 |
| Full LTF | 98.40 | 97.68 | 99.09 | 99.89 | 99.86 |
| Branch | mAcc | Real Acc | Fake Acc | AUC | AP |
| Tile level | 95.26 | 92.41 | 98.12 | 99.36 | 99.34 |
| Patch level | 97.39 | 95.24 | 99.55 | 99.88 | 99.86 |
| Hierarchy | 98.40 | 97.68 | 99.09 | 99.89 | 99.86 |
| Method | FLOPs (G) | Latency (ms) | Memory (MB) | Params (M) | Accuracy (%) |
| Single-patch methods | |||||
| UnivFD | 77.83 | 10.60 | 1699 | 427.6 | 61.99 |
| DRCT | 77.83 | 10.60 | 1664 | 428.4 | 67.17 |
| AIDE | 225.70 | 32.90 | 3425 | 897.8 | 56.46 |
| Multi-patch methods | |||||
| SPAI | 658.00 | 233.00 | 4376 | 139.95 | 69.92 |
AI-Generated Image Detection
AI-generated image detection identifies synthetic or AI-edited images and develops methods that generalize across generators, domains, and image transformations.
See all 203 papers