Digital forensic investigations often face significant challenges when recovering fragmented multimedia files that lack file system metadata. While traditional file carving relies on signatures and discriminative deep learning models for fragment classification, these methods cannot reconstruct or predict missing data. We propose a generative approach to multimedia carving using bGPT, a byte-level transformer designed for next-byte prediction. By feeding partial BMP image data into the model, we simulate the generation of likely fragment continuations. We evaluate the fidelity of these predictions using different metrics, namely, cosine similarity, structural similarity index (SSIM), chi-square distance, and Jensen-Shannon divergence (JSD). Our findings demonstrate that generative models can effectively predict byte-level patterns to support fragment matching in unallocated disk space.
Figures & tables
Figure 1: The bGPT framework simulates digital systems through native binary data and integrates diverse data types into a single model, treating everything as a byte sequence. [ 10 ] .
Figure 2: Flow diagram of a pseudo-simulation pipeline where fragmented BMP inputs are processed by bGPT to predict the next fragment, which is compared to a mixed multimedia fragment pool using similarity metrics to identify the top five matches.
Metric
Mean
Std Dev
95% CI (±)
P1 ( ∼ 1.8 kB)
P2 ( ∼ 1.2 kB)
P3 ( ∼ 0.6 kB)
P1 ( ∼ 1.8 kB)
P2 ( ∼ 1.2 kB)
P3 ( ∼ 0.6 kB)
P1 ( ∼ 1.8 kB)
P2 ( ∼ 1.2 kB)
P3 ( ∼ 0.6 kB)
Chi-square
409.58
335.61
174.97
161.23
139.24
61.96
23.12
19.96
8.88
Cosine
0.6690
0.6749
0.7641
0.1735
0.1605
0.1198
0.0249
0.0230
0.0172
JSD
0.1864
0.2338
0.2458
0.0798
0.1050
0.0938
0.0114
0.0151
0.0134
SSIM
0.0856
0.1016
0.3262
0.0738
0.0966
0.1628
0.0106
0.0138
0.0233
Table 1: Results of similarity analysis between predicted fragments and their real continuations across fragment sizes.
Figure 3: Box plots showing similarity scores between predicted and real fragments across three predicted fragment sizes: P1 (1.8 kB), P2 (1.2 kB), and P3 (0.6 kB), using four evaluation metrics: cosine similarity, SSIM, chi-square distance, and Jensen–Shannon divergence. Each plot captures the distribution and variation in similarity across different prediction groups.
Figure 4: SSIM heatmap between predicted and real image fragments from Fig. 5 . Local SSIM values were computed using a 7×7 sliding window, where each pixel represents the similarity of its local neighborhood.
Figure 5: Example of fragment reconstruction using bGPT. The input fragment (a) is provided to the model to generate the predicted continuation (b), which approximates the true continuation (c). Combining (a) and (b) produces the reconstructed image (d), compared against the original full image (e).
Byte-level hierarchical language models (LMs) have recently emerged as a robust alternative to their popular counterparts that use subword tokenization. However, generating one byte at a time remains a bottleneck for inference speed. To address this, we introduce multi-byte prediction (MBP), which generates multiple bytes in parallel, speeding up inference with minimal performance impact and no additional parameters. MBP builds on the popular multi-token prediction (MTP) paradigm with two crucial innovations. First, we introduce a variable-length prediction window that aligns with the latent tokens, or segments, of a hierarchical LM. Second, we implement a novel attention-masking scheme that enables parallel byte prediction without violating causality. We show that multi-byte prediction strikes a Pareto-optimal trade-off across multiple generative tasks, instruction following, question answering, summarization, and machine translation, achieving the best trade-off between performance and inference throughput.
Abraham Toluwase Owodunni, Chibuzor Okocha, Christan Grant +2
The Ohio State University · University of Florida · University of Washington
As generative artificial intelligence evolves, deepfake attacks have escalated from single-modality manipulations to complex, multimodal threats. Existing forensic techniques face a severe generalization bottleneck: by relying excessively on superficial, modality-specific artifacts, they neglect the shared latent forgery knowledge hidden beneath variable physical appearances. Consequently, these models suffer catastrophic performance degradation when confronted with unseen "dark modalities." To break this limitation, this paper introduces a paradigm shift that redefines multimodal forensics from conventional "feature fusion" to "modality generalization." We propose the first modality-agnostic forgery (MAF) detection framework. By explicitly decoupling modality-specific styles, MAF precisely extracts the essential, cross-modal latent forgery knowledge. Furthermore, we define two progressive dimensions to quantify model generalization: transferability toward semantically correlated modalities (Weak MAF), and robustness against completely isolated signals of "dark modality" (Strong MAF). To rigorously assess these generalization limits, we introduce the DeepModal-Bench benchmark, which integrates diverse multimodal forgery detection algorithms and adapts state-of-the-art generalized learning methods. This study not only empirically proves the existence of universal forgery traces but also achieves significant performance breakthroughs on unknown modalities via the MAF framework, offering a pioneering technical pathway for universal multimodal defense.
Jingtong Dou, Chuancheng Shi, Jian Wang +3
The University of Sydney · Nanjing University of Posts and Telecommunications · National University of Singapore
Recent generative models can produce images with few obvious visual artifacts, weakening detectors and explanations that rely only on surface appearance. We present LaP-Forensics, a multimodal framework that augments RGB semantics with reconstruction-based forensic evidence. A frozen Stable Diffusion DDIM inversion-reconstruction model provides a fixed reconstruction reference, and its residual map measures local compatibility with that reference. Independent projectors encode the RGB image and residual map before a structured Where-What-Why model predicts a textual analysis and an artifact mask. Supervised fine-tuning is followed by Group Relative Policy Optimization (GRPO), whose reward combines mask overlap with output-structure and evidence-reference terms. These text-side terms encourage the model to refer to the consistency map but do not constitute a verifier of free-form textual truth. A separate image-level head fuses RGB and DDIM-residual class features. Experiments show cross-generator detection on UniversalFakeDetect and competitive artifact localization on the official SynthScars benchmark. Controlled cue-construction, inversion-horizon, component, reward-term, and counterfactual analyses support the utility of the residual stream under the evaluated settings, while free-form textual faithfulness and reliability under post-processing remain open limitations.
Can Wang, Yuhao Wang, Yushe Cao +2
The Hong Kong Polytechnic University, Hong Kong, China · University College London, London, United Kingdom · Tsinghua University, Beijing, China +2