FISSION: Label Augmentation for Bot Detection

Sen Yang; Ignacy Nieweglowski; Aviv Yaish

cs.AIAug 14, 2026

FISSION: Label Augmentation for Bot Detection

Authors: Sen Yang, Ignacy Nieweglowski, Aviv Yaish

Organizations: Yale University, IC3 · Staples High School

Abstract

Bot accounts and coordinated influence operations are often discovered via heuristic methods, leaving a dearth of reliable ground-truth labels for training detection systems. To address this challenge, we study a natural question: can we generate labels to assist in learning embeddings in which bots and accounts from the same coordinated operation are close? We present FISSION, a method to generate labels by splitting each account's activity into positively labeled sub-accounts. Given this label source, we train detection models which preserve behavioral regularities recurring across positive sub-accounts. We evaluate FISSION and show it outperforms prior methods in detecting Wikipedia sockpuppets and Twitter/X bots.

Figures & tables

Figure 1: FISSION samples positive pairs from disjoint temporal views of the same unlabeled history. The NT-Xent objective trains encoder $f_{\theta}$ through a temporary projection head $g_{\psi}$ . Classification discards $g_{\psi}$ , retains $f_{\theta}$ , and uses labels only to fit $q_{\phi}$ .

Approach	AUROC	AUPRC / AP	F1-Score	F0.5-Score	Accuracy	Precision	Recall
Random [RD25]	$50.10\pm 0.14$	$50.85\pm 0.09$	$40.34\pm 0.11$	$36.52\pm 0.12$	$50.10\pm 0.16$	$34.46\pm 0.12$	$50.05\pm 0.15$
Majority [RD25]	-	-	-	-	$65.60\pm 0.00$	-	-
RoBERTa [RD25]	$65.70\pm 0.00$	$50.45\pm 0.03$	$57.97\pm 0.01$	$57.52\pm 0.06$	$66.98\pm 0.06$	$59.54\pm 0.13$	$67.63\pm 0.17$
Standard Enc. [RD25]	$68.33\pm 0.09$	$50.67\pm 0.33$	$60.05\pm 0.18$	$58.73\pm 0.16$	$68.90\pm 0.07$	$59.72\pm 0.32$	$69.88\pm 0.34$
Pre-trained Enc. [RD25]	$62.74\pm 0.02$	$44.80\pm 0.19$	$57.49\pm 0.13$	$52.90\pm 0.12$	$62.79\pm 0.05$	$51.45\pm 0.25$	$74.76\pm 0.28$
Reptile Enc. [RD25]	$78.98\pm 0.12$	$62.21\pm 0.08$	$67.46\pm 0.53$	$67.89\pm 0.17$	$77.51\pm 0.19$	$69.43\pm 0.26$	$70.81\pm 0.82$

Table 1: Comparison with prior work on the sockpuppet dataset. [RD25] refers to [ 41 ] .

Method	Accuracy	F1-score
[ 27 ]	97.45( $\pm$ 0.23)	96.87( $\pm$ 0.43)
[ 36 ]	96.84( $\pm$ 0.13)	97.44( $\pm$ 0.74)
[ 16 ]	97.15( $\pm$ 0.32)	97.78( $\pm$ 0.24)
[ 19 ]	96.52( $\pm$ 0.71)	97.30( $\pm$ 0.53)
[ 32 ]	98.35( $\pm$ 0.24)	98.71( $\pm$ 0.18)
[ 35 ]	98.50( $\pm$ 0.00)	98.82( $\pm$ 0.00)

Table 2: Comparison with prior work on Cresci15.

Dataset	Positive Class	Positive	Negative	Total	View Eligibility
Cresci15	Bot	3,351	1,950	5,301	At least 20 events
Wikipedia Active	Sockpuppet	66,247	95,357	161,604	At least 10 edits
Wikipedia Full	Sockpuppet	132,890	3,179,197	3,312,087	Routed by edit count

Table 3: The cohorts that on which our system is evaluated experimentally. Wikipedia’s active cohort supplies the FISSION teacher and the controlled representation analyses. Sparse histories in the full cohort are scored by FISSION -Distill.

Figure 2: Activity support in two evaluated cohorts. The median Cresci15 human has 864 released events, compared with 23 for a bot. Wikipedia is less polarized, with medians of 22 edits for benign accounts and 33 for sockpuppets. Curves are empirical complementary cumulative distributions, with each account counted once. Dashed lines mark the minimum history length for contrastive training. The Cresci15 curves cover all 5,301 profiles: the 153 profiles without released post rows are counted at zero in the empirical distribution and fall outside the logarithmic horizontal axis. Every retained Wikipedia account is eligible.

Stage	Information Used for Fitting
Feature Transforms	Complete histories in outer fold training portion, no task labels
Encoder Pretraining	Eligible fissioned training views, no task labels
Classifier Selection	Representations of labeled training accounts and inner-fold validation labels
Outer Evaluation	Complete held-out histories and labels used only for scoring

Table 4: Information available at each outer evaluation stage.

Contrast	Quantities Held Fixed
Raw vs. One Encoder	Accounts, input features, logistic classifier, folds, and threshold rule
One Fixed vs. Three Adaptive	Base features and classifier family, view rule and number of encoder-classifier pairs change
Fixed vs. Adaptive Views	Accounts, encoder count, features, and classifier
Raw vs. FISSION RBF	Accounts, input features, kernel classifier, folds, and threshold rule
Feature Removal	Protocol and architecture, one input family is removed
Label Budget	Representation, folds, and paired labeled subset, only label count changes

Table 5: Main experimental contrasts. Each row changes a component while preserving the listed parts of the evaluation.

Dataset	Model	Accuracy (%)	Positive $F_{1}$ (%)	ROC-AUC (%)
Wikipedia Full	FISSION -Distill	$\mathbf{98.684\pm 0.002}$	$\mathbf{81.957\pm 0.025}$	$\mathbf{94.543\pm 0.005}$
Cresci15	Raw Features + Logistic	$98.780\pm 0.077$	$99.023\pm 0.062$	$99.787\pm 0.034$
	FISSION , Three Encoders + Logistic	$99.136\pm 0.055$	$99.307\pm 0.044$	$99.921\pm 0.012$
	Raw Features + RBF SVM	$98.537\pm 0.127$	$98.827\pm 0.101$	$99.827\pm 0.009$
	FISSION , Three Encoders + RBF SVM	$\mathbf{99.223\pm 0.063}$	$\mathbf{99.377\pm 0.051}$	$\mathbf{99.924\pm 0.012}$
	FISSION + Profile Fallback (full cohort)	$\mathbf{99.227\pm 0.061}$	$\mathbf{99.389\pm 0.048}$	—

Table 6: Primary account-level performance. Positive

F_{1}

refers to sockpuppets on Wikipedia and bots on Cresci15. The Cresci15 rows report repeated outer evaluations: the profile-fallback row covers all 5,301 profiles and the other rows cover the 5,148 accounts with released posts. A combined Cresci15 ROC-AUC is omitted because its two routing branches are not jointly calibrated.

Figure 3: Accuracy across label budgets. Raw features and FISSION use the same labeled accounts and classifier family per point. FISSION pretraining includes every eligible outer-training history but uses none of their labels. Points show mean pooled out-of-fold accuracy. Bars show descriptive standard deviations over outer-evaluation/subset combinations.

Panel of Figure 4 — Figure 4: An illustration of the FISSION pipeline.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Cohort	Positive	Negative	Total
Cresci15 Event Cohort	3,202	1,946	5,148
Cresci15 Full Profile Cohort	3,351	1,950	5,301
Wikipedia Supplied Summary	116,846	3,195,242	3,312,088
Wikipedia Full Scoring	132,890	3,179,197	3,312,087
Wikipedia Active Cohort	66,247	95,357	161,604

Appendix

Table 7: Account counts. Positive denotes bot on Cresci15 and sockpuppet on Wikipedia.

Dataset	Train	Positive Test	Negative Test
Cresci15	4,118-4,119	640-641	389-390
Wikipedia	129,283-129,284	13,249-13,250	19,071-19,072

Appendix

Table 8: Accounts per outer fold. Total test sizes are 1,029-1,030 for Cresci15 and 32,320-32,321 for Wikipedia.

Dataset	Behavioral Groups	Lexical	Total
Cresci15	Hour 24, Day 7, Scalars 8	50	89
Wikipedia	Hour 24, Magnitude 3, Rhythm 3	50	80

Appendix

Table 9: Feature dimensions. Wikipedia magnitude contains edit-size and comment-length statistics. Rhythm contains page focus and inter-edit statistics.

Figure 5: Fixed and count-adaptive fissioning of a 25-event history. The adaptive rule forms five denser views. Both rules reach ten views at 50 events.

Component	Cresci15	Wikipedia
Eligibility	At least 20 events to form views, all accounts evaluated	Cohort and view eligibility require at least 10 edits
Views	Adaptive $m_{u}=\operatorname{clip}(\lfloor n_{u}/5\rfloor,2,10)$ , fixed ten-way comparison, all unordered pairs	Adaptive views for the complete model, fixed ten-way views for the single-encoder comparison
Input	39 behavioral + 50 lexical dimensions	30 behavioral + 50 lexical dimensions
Text Transform	Outer-training TF-IDFwith 5,000 terms and minimum document frequency 5, followed by a 50-dimensional truncated SVD.
Encoder / Projector	$d_{\mathrm{in}}$ -128-128-64 encoder with BN, ReLU, and normalized output, training-only 64-128-32 projector with ReLUand normalization.
Contrastive Fit	Adam, learning rate $10^{-3}$ , 50 epochs, 256 pairs (512 views), temperature $0.12$ , single-positive pairs, and diagonal masking.

Appendix

Table 10: Configurations for the main experiments. Hyperparameters and thresholds are reselected within every outer fold.

Approach	AUROC	AUPRC / AP	F1-Score	F0.5-Score	Accuracy	Precision	Recall
Reptile Enc. [RD25]	$78.98\pm 0.12$	$62.21\pm 0.08$	$67.46\pm 0.53$	$67.89\pm 0.17$	$77.51\pm 0.19$	$69.43\pm 0.26$	$70.81\pm 0.82$
Our Work (Active)	$98.917\pm 0.013$	$98.437\pm 0.013$	$94.307\pm 0.014$	$93.992\pm 0.030$	$95.306\pm 0.013$	$93.783\pm 0.044$	$94.838\pm 0.039$
Our Work (Full)	$94.543\pm 0.005$	$84.084\pm 0.012$	$81.957\pm 0.025$	$87.182\pm 0.057$	$98.684\pm 0.002$	$91.052\pm 0.084$	$74.515\pm 0.027$

Appendix

Table 11: Wikipedia results in the column order of the main-text comparison in Table 1 . The RD25 Reptile baseline and the full-dataset FISSION -Distill result are repeated from the main-text table. The 161,604-account active-cohort row shows the teacher model from which the sparse-history students are distilled. RD25 reports AUPRC, the active row uses trapezoidal AUPRC, and the full row uses Average Precision (AP).

Dataset	Contrast	$\Delta$ Acc.	95% CI
Wikipedia	Raw $\rightarrow$ One Encoder	$+3.181$	$[3.069,3.294]$
	One $\rightarrow$ Three Adaptive	$+0.857$	$[0.804,0.910]$
	Fixed $\rightarrow$ Adaptive Views	$+0.299$	$[0.188,0.411]$
	Remove Rhythm	$-10.413$	$[-10.583,-10.245]$
	Remove Text	$-0.549$	$[-0.619,-0.477]$
Cresci15	Raw $\rightarrow$ One Encoder	$+0.043$	$[-0.128,0.208]$

Appendix

Table 12: Principal architecture and feature contrasts in accuracy percentage points. The Wikipedia view-rule comparison uses a separate 30,000-account cohort, while all other rows use the main evaluated cohorts.

Views	Classifier	Accuracy (%)	Bot $F_{1}$ (%)
Fixed Ten-Way	Logistic	$99.007\pm 0.050$	$99.205\pm 0.040$
Count-Adaptive	Logistic	$99.136\pm 0.055$	$99.307\pm 0.044$
Fixed Ten-Way	RBF SVM	$99.217\pm 0.061$	$99.372\pm 0.049$
Count-Adaptive	RBF SVM	$\mathbf{99.223\pm 0.063}$	$\mathbf{99.377\pm 0.051}$

Appendix

Table 13: Cresci15 fissioning-by-classifier comparison. Adaptive minus fixed accuracy is

+0.128

points with logistic regression (95% CI

[+0.049,+0.210]

) and

+0.006

with the RBF SVM (

[-0.047,+0.060]

Variant	Acc. (%)	$\Delta$ (pp)	95% CI	Holm $p$
All Features	$99.242\pm 0.066$	-	-	-
Without Hour	99.211	$-0.031$	$[-0.120,+0.060]$	0.73
Without Day	99.180	$-0.062$	$[-0.150,+0.020]$	0.50
Without Scalars	99.176	$-0.066$	$[-0.210,+0.070]$	0.73
Without Text	97.296	$\mathbf{-1.946}$	$\mathbf{[-2.323,-1.585]}$	$\mathbf{0.0004}$
Text Only	99.083	$-0.159$	$[-0.310,-0.020]$	-

Appendix

Table 14: Cresci15 leave-one-group-out feature analysis. The all-feature row is the separately fitted paired baseline for this ablation. The Holm family contains the four removal variants, and text-only is an additional reference point.

Variant	Accuracy (%)	Change (pp)	95% CI (pp)
All Features	95.314	-	-
Without Rhythm	84.902	$-10.413$	$[-10.583,-10.245]$
Without Text	94.765	$-0.549$	$[-0.619,-0.477]$
Text Only	77.991	$-17.323$	$[-17.516,-17.129]$

Appendix

Table 15: Wikipedia feature-block analysis. The all-feature row is the separately fitted paired baseline for this ablation. Negative changes denote lower accuracy.

Classifier	Search Grid and Fixed Settings
Logistic Regression	$C\in\{0.01,0.1,1,10,100\}$ , ordinary or balanced class weights, LM-BFGS, at most 1,000 iterations
RBF SVM	$C\in\{1,10,100\}$ , $\gamma\in\{\mathrm{scale},0.1\}$ , Platt map fit to inner out-of-fold decision values
XGBoost	trees $\in\{200,500\}$ , depth $\in\{2,3,4\}$ , learning rate $\in\{0.05,0.1\}$
LightGBM	trees $\in\{200,500\}$ , depth $\in\{2,3,4\}$ , learning rate $\in\{0.05,0.1\}$
MLP	one hidden layer of width 32 or 128, $\alpha\in\{10^{-4},10^{-2}\}$ , LM-BFGS, at most 1,000 iterations
Random Forest	500 trees, maximum features $\sqrt{d}$ or $0.5d$

Appendix

Table 16: Classifier-family settings. Tree row and column subsampling equal one unless listed otherwise.

Classifier and Layout	Accuracy (%)	$\Delta$ Acc. (pp)
TabPFN, Per Encoder	99.219	$+0.062$
RBF SVM, Per Encoder	99.219	$+0.062$
TabPFN, Joint	99.207	$+0.050$
RBF SVM, Joint	99.200	$+0.043$
XGBoost, Per Encoder	99.184	$+0.027$
Logistic, Joint (Reference)	99.157	$0$

Appendix

Table 17: Cresci15 classifier comparison on fixed representations, separate from the repeated end-to-end evaluation in Table 6 . Accuracy differences are measured against logistic regression on the concatenated representation.

Figure 6: Positive-class

F_{1}

(top) and ROC-AUC (bottom) across label budgets for Cresci15 (left) and Wikipedia (right). Orange dotted lines denote raw features and blue solid lines denote FISSION , while bars show sample standard deviations.

Budget	Labels per Fold	Input	Accuracy (%)	Positive $F_{1}$ (%)	ROC-AUC (%)
1%	41	Raw	$94.287\pm 1.056$	$95.402\pm 0.872$	$96.819\pm 1.087$
		FISSION	$97.091\pm 0.817$	$97.640\pm 0.689$	$99.685\pm 0.076$
2%	82	Raw	$95.887\pm 0.585$	$96.698\pm 0.476$	$98.309\pm 0.414$
		FISSION	$97.929\pm 0.318$	$98.332\pm 0.263$	$99.775\pm 0.042$
5%	206	Raw	$97.032\pm 0.327$	$97.614\pm 0.269$	$99.181\pm 0.164$
		FISSION	$98.487\pm 0.163$	$98.786\pm 0.132$	$99.832\pm 0.027$

Appendix

Table 18: Complete Cresci15 label-efficiency results with the RBF SVM.

Budget	Labels per Fold	Input	Accuracy (%)	Positive $F_{1}$ (%)	ROC-AUC (%)
1%	1,293	Raw	$88.464\pm 0.657$	$86.332\pm 0.645$	$95.226\pm 0.345$
		FISSION	$94.577\pm 0.098$	$93.445\pm 0.111$	$98.649\pm 0.024$
2%	2,586	Raw	$89.801\pm 0.529$	$87.861\pm 0.522$	$95.985\pm 0.239$
		FISSION	$94.895\pm 0.079$	$93.826\pm 0.074$	$98.774\pm 0.020$
5%	6,464	Raw	$90.738\pm 0.317$	$88.949\pm 0.323$	$96.412\pm 0.136$
		FISSION	$95.153\pm 0.046$	$94.129\pm 0.052$	$98.857\pm 0.016$

Appendix

Table 19: Complete Wikipedia label-efficiency results with logistic regression.

Budget	Labels per Fold	Input	Accuracy (%)	Positive $F_{1}$ (%)	ROC-AUC (%)
1%	41	Raw	$93.118\pm 0.823$	$94.352\pm 0.711$	$96.294\pm 0.736$
		FISSION	$96.646\pm 0.782$	$97.274\pm 0.664$	$97.720\pm 1.932$
2%	82	Raw	$95.141\pm 0.493$	$96.061\pm 0.415$	$97.748\pm 0.466$
		FISSION	$97.563\pm 0.513$	$98.035\pm 0.422$	$98.370\pm 2.298$
5%	206	Raw	$96.646\pm 0.320$	$97.298\pm 0.258$	$98.826\pm 0.248$
		FISSION	$98.347\pm 0.236$	$98.674\pm 0.191$	$99.668\pm 0.104$

Appendix

Table 20: Cresci15 label efficiency with logistic regression.

Figure 7: Row-normalized out-of-fold confusion matrices for Cresci15 (left) and Wikipedia (right), averaged over the main repeated evaluations. Cells show mean within-class percentages with sample standard deviations.

Figure 8: Event-count ECDFsfor all Cresci15 accounts, accounts misclassified at least once, and accounts misclassified in every repeated evaluation. The dashed line marks the 20-event contrastive-eligibility threshold.

Figure 9: Cliff’s

\delta

for directly computed scalar features on balanced samples of 1,000 accounts per class: Cresci15 (left) and Wikipedia (right). Positive values indicate larger values for bots or sockpuppets. Fitted lexical coordinates are omitted.

Figure 10: Two-dimensional PCA summaries [ 29 ] of raw inputs and concatenated three-encoder representations in one prespecified outer-test fold per dataset: the first fold of the first reported repetition, containing 1,030 Cresci15 accounts and a class-balanced sample of 2,000 Wikipedia accounts. Each projection is fitted separately on the corresponding training features and applied to the displayed test accounts.

Social Bot Detection

See all 12 papers

Oct 1, 2026

Social bot detection in the age of ChatGPT: Challenges and opportunities

We present a comprehensive overview of the challenges and opportunities in social bot detection in the context of the rise of sophisticated AI-based chatbots. By examining the state of the art in social bot detection techniques and the more salient real-world application to date, we identify gaps and emerging trends in the field, with a focus on addressing the unique challenges posed by AI-generated conversations and behaviors. We suggest potentially promising opportunities and research directions in social bot detection, including (i) the use of generative agents for synthetic data generation, testing and evaluation; (ii) the need for multimodal and cross-platform detection based on network and behavioral signatures of coordination and influence; (iii) the opportunity to extend bot detection to non-English and low-resource language settings; and, (iv) the room for development of collaborative, federated learning detection models that can help facilitate cooperation between different organizations and platforms while preserving user privacy.

Emilio Ferrara

Social Bot Detection Social Media Analysis AI-Generated Text Detection

Ask PDF

Jul 29, 2026

What Does It Take to Detect an AI Agent? Minimal Feature Sets for Behavioral Detection under Browser Automation

Bot detectors deployed at scale treat traffic as binary: human or bot. This assumption breaks when AI agents browse the web through browser automation, a traffic class that is neither and that binary classifiers structurally cannot represent. We present a three-class detection framework distinguishing humans, bots, and AI agents, and show that the binary-vs-agent confusion is architectural: a binary human-vs-bot detector misroutes agent sessions because its label space lacks an agent class. On our controlled benchmark, an MLP binary classifier misclassifies 39.1% of real AI agents as human and a SAINT binary transformer misclassifies 34.5%; adding an explicit agent class yields per-class agent F1 = 1.000 in all 30 runs (3 model families

\times

10 seeds). To measure evasion resistance, we construct a five-level evasion ladder spanning passive observation, GAN-generated trajectories, and replay of real human cursor data (

n = 2299

evasion sessions). Across 10 seeds and 3 model families we observe zero agent misses in 22990 per-seed predictions. The discriminative signal is a browser-automation artifact, not evidence of agent reasoning: Playwright does not emit the raw pointer-move and wheel-delta streams a physical input device produces, and this absence signature survives trajectory manipulation. Exhaustive search over all feature subsets of size 1-5 (9401 GBMs) shows that two behavioral features (mouse_event_rate, teleport_click_ratio) give 100% observed agent recall at every evasion level with agent precision 0.994; five features lift macro-F1 to 0.991. The signal is redundantly encoded: removing teleport_click_ratio leaves agent detection at 100%. The single-feature regime is degenerate, flagging every agent only by collapsing the classifier to always predict "agent". Two features robustly isolate agents; five separate all three traffic classes at macro-F1

\geq 0.99

Vishisht Choudhary, Lukas Schmidt, Anne Zoë Kenntner +3

Technical University of Munich Munich, Germany · Kontext Munich, Germany

AI Agent Security Feature Selection

Ask PDF

Apr 2, 2026

TRACE-Bot: Detecting Emerging LLM-Driven Social Bots via Implicit Semantic Representations and AIGC-Enhanced Behavioral Patterns

Large Language Model-driven (LLM-driven) social bots pose a growing threat to online discourse by generating human-like content that evades conventional detection. Existing methods suffer from limited detection accuracy due to overreliance on single-modality signals, insufficient sensitivity to the specific generative patterns of Artificial Intelligence-Generated Content (AIGC), and a failure to adequately model the interplay between linguistic patterns and behavioral dynamics. To address these limitations, we propose TRACE-Bot, a unified dual-channel framework that jointly models implicit semantic representations and AIGC-enhanced behavioral patterns. TRACE-Bot constructs fine-grained representations from heterogeneous sources, including personal information data, interaction behavior data and tweet data. A dual-channel architecture captures linguistic representations via a pretrained language model and behavioral irregularities via multidimensional activity features augmented with signals from state-of-the-art (SOTA) AIGC detectors. The fused representations are then classified through a lightweight prediction head. Experiments on two public LLM-driven social bot datasets demonstrate SOTA performance, achieving accuracies of 98.46% and 97.50%, respectively. The results further indicate strong robustness against advanced bot strategies, highlighting the effectiveness of jointly leveraging implicit semantic representations and AIGC-enhanced behavioral patterns for emerging LLM-driven social bot detection.

Zhongbo Wang, Zhiyu Lin, Zhu Wang +1

School of Cyber Science and Engineering, Sichuan University, Sichuan 610207, China · School of Software and Microelectronics, Peking University, Beijing 102600, China · College of Computer Science, Sichuan University, Sichuan 610065, China +1

Social Bot Detection LLM Security

Ask PDF

Abstract

Social Bot Detection

Explore similar work

Social bot detection in the age of ChatGPT: Challenges and opportunities

What Does It Take to Detect an AI Agent? Minimal Feature Sets for Behavioral Detection under Browser Automation

TRACE-Bot: Detecting Emerging LLM-Driven Social Bots via Implicit Semantic Representations and AIGC-Enhanced Behavioral Patterns