TRACE-Bot: Detecting Emerging LLM-Driven Social Bots via Implicit Semantic Representations and AIGC-Enhanced Behavioral Patterns
Authors: Zhongbo Wang, Zhiyu Lin, Zhu Wang, Haizhou Wang
Organizations: School of Cyber Science and Engineering, Sichuan University, Sichuan 610207, China · School of Software and Microelectronics, Peking University, Beijing 102600, China · College of Computer Science, Sichuan University, Sichuan 610065, China · Law School, Sichuan University, Sichuan 610207, China
Large Language Model-driven (LLM-driven) social bots pose a growing threat to online discourse by generating human-like content that evades conventional detection. Existing methods suffer from limited detection accuracy due to overreliance on single-modality signals, insufficient sensitivity to the specific generative patterns of Artificial Intelligence-Generated Content (AIGC), and a failure to adequately model the interplay between linguistic patterns and behavioral dynamics. To address these limitations, we propose TRACE-Bot, a unified dual-channel framework that jointly models implicit semantic representations and AIGC-enhanced behavioral patterns. TRACE-Bot constructs fine-grained representations from heterogeneous sources, including personal information data, interaction behavior data and tweet data. A dual-channel architecture captures linguistic representations via a pretrained language model and behavioral irregularities via multidimensional activity features augmented with signals from state-of-the-art (SOTA) AIGC detectors. The fused representations are then classified through a lightweight prediction head. Experiments on two public LLM-driven social bot datasets demonstrate SOTA performance, achieving accuracies of 98.46% and 97.50%, respectively. The results further indicate strong robustness against advanced bot strategies, highlighting the effectiveness of jointly leveraging implicit semantic representations and AIGC-enhanced behavioral patterns for emerging LLM-driven social bot detection.
Figures & tables
Fig. 1: The proposed social bot detection model in this paper.
Dataset
Category
User Count
Tweet Count
Fox8-23
Genuine Users
1,140
218,245
Social Bots
1,140
149,783
BotSim-24
Genuine Users
1,000
129,722
Social Bots
1,906
129,728
TABLE I: Dataset Statistics
Category
Model
Fox8-23
BotSim-24
ACC
PRE
REC
F1
ACC
PRE
REC
F1
Traditional Machine Learning
Pasricha et al. [ 38 ]
0.8202
0.8687
0.7544
0.8075
0.5300
0.5337
0.4500
0.4891
BotRuler [ 40 ]
0.5424
0.5223
0.9990
0.6860
0.5000
0.5000
0.9990
0.6664
Deep Learning
Arin et al. [ 43 ]
0.9627
0.9451
0.9825
0.9634
0.8150
0.8247
0.8000
0.8122
Mou et al. [ 39 ]
0.9620
0.9489
0.9766
0.9625
0.6300
0.6186
0.8210
0.7050
Graph Neural Networks
BotRGCN1 [ 41 ]
0.6754
0.5894
0.9953
0.7404
0.5675
0.8571
0.0957
0.1722
TABLE II: Performance of Different Social Bot Detection Methods on Fox-8-23 and BotSim-24
Module Settings
ACC
PRE
REC
F1
TRACE-Bot
0.9846
0.9825
0.9868
0.9847
- Textual Channel
0.9561
0.9483
0.9649
0.9565
- Behavioral Channel
0.9189
0.8687
0.9868
0.9240
- Textual Channel and Behavioral Channel
0.9474
0.9359
0.9605
0.9481
TABLE III: Results of Module Ablation Study of Our Method
Modality Settings
Fox8-23
BotSim-24
ACC
PRE
REC
F1
ACC
PRE
REC
F1
TRACE-Bot
0.9846
0.9825
0.9868
0.9847
0.9750
0.9567
0.9950
0.9755
- Personal Information Data
0.9276
0.9372
0.9167
0.9268
0.7825
0.7756
0.7950
0.7852
- Interaction Behavior Data
0.9737
0.9655
0.9825
0.9739
0.9424
0.8968
0.9990
0.9455
- Tweet Data
0.9803
0.9740
0.9868
0.9804
0.9845
0.9708
0.9990
0.9848
- Personal Information Data and Interaction Behavior Data
0.8224
0.7795
0.8991
0.8350
0.7950
0.7950
0.7950
0.7950
TABLE IV: Results of Modality Ablation Study of Our Method
Fig. 2: Results of representation learning study of our method on Fox8-23.
Fig. 3: Results of representation learning study of our method on BotSim-24.
Fig. 4: Results of label efficiency study of our method.
Fig. 5: Results of robustness study of our method on Fox8-23 and BotSim-24.
Fig. 6: Results of hyperparameter study of our method on Fox8-23 and BotSim-24.
Model Setting
ACC
PRE
REC
F1
BERT-base
0.9737
0.9779
0.9693
0.9736
RoBERTa-base
0.9496
0.9362
0.9649
0.9503
DistilGPT-2
0.9320
0.9487
0.9781
0.9350
DeBERTa-v3
0.9605
0.9487
0.9737
0.9610
GPT-2 (Proposed)
0.9846
0.9825
0.9868
0.9847
TABLE V: Result of Comparison of Pretrained Language Models
Prompt Setting
ACC
PRE
REC
F1
“{description} {location} {name}”
0.9605
0.9526
0.9693
0.9609
“This user’s profile: name is {name}, located in {location}, and describes themselves as: {description}.”
0.9671
0.9610
0.9737
0.9673
“Classify whether this social media account is a bot or human based on the following profile information: Name: {name}, Location: {location}, Profile: {combined_text}”
0.9583
0.9524
0.9649
0.9586
“Profile: {combined_text}”
0.9605
0.9487
0.9737
0.9610
No Prompt (Proposed)
0.9846
0.9825
0.9868
0.9847
TABLE VI: Result of Comparison of Prompt Encoding Strategies
Universitat Pompeu Fabra, Barcelona, Spain · Observatory on Social Media, Indiana University, Bloomington, Indiana, USA · KTH Royal Institute of Technology, Stockholm, Sweden