FIBER: Bayesian Fusion of Fast-DetectGPT and IRT-Informed Detector Evidence for LLM-Generated Text Detection
Organizations: School of Mathematics and Statistics, Shanghai Polytechnic University, 2360 Jinhai Road, Pudong New District, 201209, Shanghai, China. · School of Computing and Artificial Intelligence, Fuyao University of Science and Technology, No. 104 Zhihui Avenue, Fuzhou High-tech Industrial Development Zone, 350109, Fujian, China. · School of Computer and Information Engineering, Shanghai Polytechnic University, 2360 Jinhai Road, Pudong New District, 201209, Shanghai, China.
Abstract
Detecting large language model (LLM)-generated text remains challenging under domain and generator shifts. Supervised detectors may generalize poorly, while zero-shot methods often rely on individual statistical signals without leveraging historical detector behavior. We propose FIBER (Fast-DetectGPT with IRT-informed Bayesian Evidence Recalibration), which integrates continuous Fast-DetectGPT scores with auxiliary detector outputs through Bayesian evidence fusion. Item response theory (IRT) models historical detector-correctness patterns, providing class-conditional detector-output likelihoods through marginalization over latent text representations. These likelihoods are combined with Fast-DetectGPT score densities and a historical class prior for prediction. When historical Fast-DetectGPT scores are unavailable, a variant called FIBER-Stream estimates class-specific score distributions from an unlabeled batch. On a 32,000-text multi-domain benchmark, FIBER achieves the highest accuracy (0.620) and improves mean AUC over Fast-DetectGPT (0.664 versus 0.600) under five-fold cross-validation. FIBER also consistently improves AUC across four individual domains. Under out-of-distribution transfer, FIBER-Stream achieved an AUC of 0.911, compared with 0.889 for FIBER and 0.922 for Fast-DetectGPT alone. These results demonstrate the potential of integrating complementary detector evidence and historical information to improve detection performance across heterogeneous settings.
Figures & tables
| Detectors | Accuracy | Precision | Recall | F1 score |
|---|---|---|---|---|
| FIBER | 0.620 (1) | 0.642 (9) | 0.542 (6) | 0.587 (4) |
| FIBER-Fast | 0.619 (2) | 0.647 (8) | 0.531 (7) | 0.582 (6) |
| FIBER-Stream | 0.606 (5) | 0.616 (14) | 0.565 (4) | 0.589 (3) |
| FIBER-Stream-Fast | 0.606 (5) | 0.618 (13) | 0.558 (5) | 0.586 (5) |
| Binoculars | 0.583 (13) | 0.676 (4) | 0.321 (15) | 0.435 (15) |
| DetectGPT | 0.618 (3) | 0.614 (15) | 0.632 (2) | 0.623 (1) |
| Domain | arXiv | Writing Prompt | ||||||
|---|---|---|---|---|---|---|---|---|
| Detector | Accuracy | Precision | Recall | F1 score | Accuracy | Precision | Recall | F1 score |
| FIBER | 0.751 | 0.759 | 0.735 | 0.747 | 0.633 | 0.712 | 0.448 | 0.549 |
| FIBER-Fast | 0.745 | 0.754 | 0.727 | 0.740 | 0.623 | 0.705 | 0.424 | 0.529 |
| FIBER-Stream | 0.707 | 0.712 | 0.699 | 0.704 | 0.638 | 0.708 | 0.471 | 0.565 |
| FIBER-Stream-Fast | 0.706 | 0.696 | 0.735 | 0.714 | 0.631 | 0.708 | 0.447 | 0.547 |
| Binoculars | 0.621 | 0.660 | 0.505 | 0.571 | 0.634 | 0.700 | 0.469 | 0.562 |
| Detector | Accuracy | Precision | Recall | F1 score |
|---|---|---|---|---|
| FIBER | 0.827 | 0.888 | 0.768 | 0.824 |
| FIBER-Fast | 0.811 | 0.818 | 0.823 | 0.821 |
| FIBER-Stream | 0.840 | 0.885 | 0.800 | 0.840 |
| FIBER-Stream-Fast | 0.827 | 0.839 | 0.831 | 0.835 |
| Binoculars | 0.859 | 0.903 | 0.820 | 0.860 |
| DetectGPT | 0.633 | 0.641 | 0.688 | 0.663 |
AI-Generated Text Detection
AI-generated text detection develops methods to distinguish human-written, machine-generated, and AI-edited text.
See all 185 papers