cs.CLMar 5, 2026

Detection of Illicit Content on Online Marketplaces using Large Language Models

Authors: Quoc Khoa Tran, Thanh Thi Nguyen, Campbell Wilson

Organizations: AiLECS Lab, Monash University, Melbourne, Australia

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Online marketplaces, while revolutionizing global commerce, have inadvertently facilitated the proliferation of illicit activities, including drug trafficking, counterfeit sales, and cybercrimes. Traditional content moderation methods such as manual reviews and rule-based automated systems struggle with scalability, dynamic obfuscation techniques, and multilingual content. Conventional machine learning models, though effective in simpler contexts, often falter when confronting the semantic complexities and linguistic nuances characteristic of illicit marketplace communications. This research investigates the efficacy of Large Language Models (LLMs), specifically Meta's Llama 3.2 and Google's Gemma 3, in detecting and classifying illicit online marketplace content using the multilingual DUTA10K dataset. Employing fine-tuning techniques such as Parameter-Efficient Fine-Tuning (PEFT) and quantization, these models were systematically benchmarked against a foundational transformer-based model (BERT) and traditional machine learning baselines (Support Vector Machines and Naive Bayes). Experimental results reveal a task-dependent advantage for LLMs. In binary classification (illicit vs. non-illicit), Llama 3.2 demonstrated performance comparable to traditional methods. However, for complex, imbalanced multi-class classification involving 40 specific illicit categories, Llama 3.2 significantly surpassed all baseline models. These findings offer substantial practical implications for enhancing online safety, equipping law enforcement agencies, e-commerce platforms, and cybersecurity specialists with more effective, scalable, and adaptive tools for illicit content detection and moderation.

Figures & tables

ReferencesData sizeFeaturesBest modelResult
[ 7 ]6,836 docsBOW & TF-IDFLogistic Regression96.60% Accuracy
[ 9 ]7,000 docsBOW & TF-IDFStochastic Gradient Descent (SGD)84.57% Accuracy
[ 21 ]1,000 docsTF-IDF, Glasgow & EntropySVM93.00% Accuracy
[ 22 ]7,997 imagesSIFT & SURFKNN82.26% Accuracy
[ 23 ]4,851 docsTF-IDFNaive Bayes93.50% Accuracy
[ 24 ]248,359 market listingsWord-based featuresMALLET Topic Modeling81.5% Precision
TABLE I: A summary of related studies in detecting online illicit content
Fig. 1: Top 10 Main Categories in the DUTA10K Dataset.
Fig. 2: Llama and Gemma training pipeline for text classification.
Fig. 3: LLM structure in binary and multi-class text classification tasks.
Model TypeLLMBaseline Model
ModelGemmaLlamaNaive BayesSVMBERT
ResultMacroWeightedMacroWeightedMacroWeightedMacroWeightedMacroWeighted
Precision0.730.810.870.890.820.850.880.90.920.86
Recall0.680.830.770.890.690.860.770.90.570.84
F10.70.820.80.880.730.850.810.890.580.78
Accuracy0.830.890.860.90.84
TABLE II: A comparison between LLMs and baseline models in binary classification
Model TypeLLMBaseline Model
ModelGemmaLlamaNaive BayesSVMBERT
ResultMacroWeightedMacroWeightedMacroWeightedMacroWeightedMacroWeighted
Precision0.620.680.680.750.230.530.560.780.340.6
Recall0.510.680.580.740.160.540.40.720.370.68
F10.540.660.610.730.160.480.440.720.340.63
Accuracy0.680.740.540.720.68
TABLE III: A comparison between LLMs and baseline models in multi-class classification

Explore similar work

CardsList
Apr 8, 2026

Making MLLMs Blind: Adversarial Smuggling Attacks in MLLM Content Moderation

Multimodal Large Language Models (MLLMs) are increasingly being deployed as automated content moderators. Within this landscape, we uncover a critical threat: Adversarial Smuggling Attacks. Unlike adversarial perturbations (for misclassification) and adversarial jailbreaks (for harmful output generation), adversarial smuggling exploits the Human-AI capability gap. It encodes harmful content into human-readable visual formats that remain AI-unreadable, thereby evading automated detection and enabling the dissemination of harmful content. We classify smuggling attacks into two pathways: (1) Perceptual Blindness, disrupting text recognition; and (2) Reasoning Blockade, inhibiting semantic understanding despite successful text recognition. To evaluate this threat, we constructed SmuggleBench, the first comprehensive benchmark comprising 1,700 adversarial smuggling attack instances. Evaluations on SmuggleBench reveal that both proprietary (e.g., GPT-5) and open-source (e.g., Qwen3-VL) state-of-the-art models are vulnerable to this threat, producing Attack Success Rates (ASR) exceeding 90%. By analyzing the vulnerability through the lenses of perception and reasoning, we identify three root causes: the limited capabilities of vision encoders, the robustness gap in OCR, and the scarcity of domain-specific adversarial examples. We conduct a preliminary exploration of mitigation strategies, investigating the potential of test-time scaling (via CoT) and adversarial training (via SFT) to mitigate this threat. Our code is publicly available at https://github.com/zhihengli-casia/smugglebench.

Zhiheng Li, Zongyang Ma, Yuntong Pan +8

State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA · School of Artificial Intelligence, University of Chinese Academy of Sciences · Beijing Key Laboratory of Super Intelligent Security of Multi-Modal Information +4

Multimodal Content ModerationAdversarial Attacks on VLMsContent Moderation
AskPDF
Aug 22, 2026

No One Model Catches Every Harm: Benchmarking Content Moderation Across Safety Scenarios

Large Language Models (LLMs) are increasingly deployed in real-world applications, yet they remain vulnerable to generating harmful content. From adversarial jailbreaks that bypass safety filters to implicit hate that evades detection, the range of risks these models pose continues to grow. While both specialized content moderators and general-purpose LLMs are being used as safety layers, the question of which model is best suited for which type of harmful content remains unanswered. We present the most comprehensive evaluation of LLM safety capabilities to date, systematically testing 53 models across 11 datasets that we organize into four distinct categories. Our evaluation under both prompt-only and prompt-response settings uncovers critical blind spots: large frontier models that lead on one category fall significantly behind smaller, specialized alternatives on others, and real-world conversational safety remains largely unsolved across all model families. These findings challenge the assumption that scale alone ensures safety, and provide the community with a structured framework for informed model selection.

Afshin Orojlooyjadid, Hitesh Patel

Oracle AI

LLM Safety EvaluationLLM Safety BenchmarksContent ModerationLLM Safety Monitoring
AskPDF
Mar 15, 2026

Tool-MCoT: Tool Augmented Multimodal Chain-of-Thought for Content Safety Moderation

The growth of online platforms and user content requires strong content moderation systems that can handle complex inputs from various media types. While large language models (LLMs) are effective, their high computational cost and latency present significant challenges for scalable deployment. To address this, we introduce Tool-MCoT, a small language model (SLM) fine-tuned for content safety moderation leveraging external framework. By training our model on tool-augmented chain-of-thought data generated by LLM, we demonstrate that the SLM can learn to effectively utilize these tools to improve its reasoning and decision-making. Our experiments show that the fine-tuned SLM achieves significant performance gains. Furthermore, we show that the model can learn to use these tools selectively, achieving a balance between moderation accuracy and inference efficiency by calling tools only when necessary.

Shutong Zhang, Dylan Zhou, Yinxiao Liu +3

Stanford University, USA · Google LLC, USA

Multimodal Content ModerationContent ModerationSmall Language ModelsLLM Tool Use
AskPDF