CARO: Chain-of-Analogy Reasoning Optimization for Robust Content Moderation

Bingzhe Wu; Haotian Lu; Yuchen Mou

cs.AIApr 12, 2026

CARO: Chain-of-Analogy Reasoning Optimization for Robust Content Moderation

Authors: Bingzhe Wu, Haotian Lu, Yuchen Mou

Organizations: Shenzhen University · Tsinghua University · National University of Singapore

Abstract

Current large language models (LLMs), even those explicitly trained for reasoning, often struggle with ambiguous content moderation cases due to misleading "decision shortcuts" embedded in context. Inspired by cognitive psychology insights into expert moderation, we introduce \caro (Chain-of-Analogy Reasoning Optimization), a novel two-stage training framework to induce robust analogical reasoning in LLMs. First, \caro bootstraps analogical reasoning chains via retrieval-augmented generation (RAG) on moderation data and performs supervised fine-tuning (SFT). Second, we propose a customized direct preference optimization (DPO) approach to reinforce analogical reasoning behaviors explicitly. Unlike static retrieval methods, \caro dynamically generates tailored analogical references during inference, effectively mitigating harmful decision shortcuts. Extensive experiments demonstrate that \caro substantially outperforms state-of-the-art reasoning models (DeepSeek R1, QwQ), specialized moderation models (LLaMA Guard), and advanced fine-tuning and retrieval-augmented methods, achieving an average F1 score improvement of 24.9% on challenging ambiguous moderation benchmarks.

Figures & tables

Figure 1: Comparison between standard reasoning and analogical reasoning paradigm on a real harmless sample.

Dataset	Qwen2.5-7B-Instruct $\rightarrow$ CarO (Ours)
Dataset	Pornography	Violence	Bias	Harmless	Average
Aegis (In-Distribution)	39.6 $\rightarrow$ 75.0	43.8 $\rightarrow$ 69.1	66.4 $\rightarrow$ 78.7	89.8 $\rightarrow$ 92.3	78.7 $\rightarrow$ 87.1
OpenAI (Out-of-Distribution)	75.3 $\rightarrow$ 82.6	23.0 $\rightarrow$ 32.3	42.1 $\rightarrow$ 44.1	81.7 $\rightarrow$ 84.0	70.8 $\rightarrow$ 74.2
Toxic-Chat (Out-of-Distribution)	59.0 $\rightarrow$ 60.8	17.6 $\rightarrow$ 42.2	–	96.9 $\rightarrow$ 97.7	93.3 $\rightarrow$ 95.0

Table 1: CarO on other benchmarks. “-” indicates that the dataset does not contain the category.

RAG-SFT	Recheck	DPO	F1 score	CoA Ratio(%)
–	–	–	64.3	0.0
✓	–	–	85.5(+21.2)	89.5(+89.5)
✓	✓	–	88.8(+3.3)	93.5(+4.0)
✓	✓	✓	89.2(+0.4)	99.3(+5.8)

Table 2: Ablation study on different strategies.

Category	Model	Politics	Pornography	Violence	Bias	Gambling	Harmless	Average
General LLMs	Qwen2.5-7B-Instruct Yang et al. (2024)	54.9	81.9	70.0	60.1	84.3	48.8	64.3
	LLaMA3-8B	58.5	55.9	81.0	65.2	90.6	44.2	67.5
	GPT-4 Achiam et al. (2023)	58.6	88.7	79.8	64.3	92.7	56.8	72.3
	Qwen2.5-32B-Instruct	59.1	91.1	84.4	67.9	95.4	54.2	74.3
	QwQ-32B	75.4	69.6	72.0	60.7	84.9	54.6	69.1
	DeepSeek V3	79.0	90.3	89.8	70.5	95.0	62.5	80.3

Table 3: Comparison of General-Purpose LLMs, Specialized Moderation LLMs, and various post-training methods.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Category	Total	Train	Test
Politics	4,378	1,200	250
Pornography	2,519	1,200	250
Violence	1,472	1,200	250
Bias	1,978	1,200	250
Gambling	2,079	1,200	250
Harmless	2,129	1,200	250

Appendix

Table 4: Dataset Splits and Distribution

Sentence	Label	Reference ( CarO generated)	Reference(Qwen2.5-7B-Instruct)
I feel that whenever Africa is mentioned, it is instinctively linked to poverty and disease , and Dubai is associated with wealth . It’s similar to how foreigners immediately think of Chinese Kung Fu and giant pandas upon hearing about China.	Bias	" Black people are all lazy ", " Indians are all liars ", " Regional discrimination against Northeast people "	"At that time, Asians inspired Africa and Latin America . In Egypt, yellow - skinned people were respected for defeating white - skinned people."
Just talk about the roommate. Why bring up the roommate from Inner Mongolia? What’s the point? This can easily come across as overgeneralizing and unfriendly…	Harmless	"Why don’t you say it’s from Guizhou?", "Isn’t it from Jiangxi?"	"There are many black intermediaries and annoying second-landlords from Northeast China in Beijing", "Aren’t people from Northeast China generally straightforward? Why is it like this…?"
The accent of the Chinese Korean ethnic group is closer to the Korean language of North Korea rather than that of South Korea, right [Surprise]	Harmless	"The Chinese pronunciation of North Koreans is related to the Korean language"	"Don’t be overly feminist about everything", " South Korean international students have extremely low quality ", "Some South Koreans indeed have extremely low quality"
The Anti Bar leans to the left. Why are there so many posts in the Anti Bar that look down on black people ? Isn’t discriminating against black people a typical characteristic of the right wing?	Bias	"Why are black people discriminated against?", "Black people are all lazy "	"At that time, Asians inspired Africa and Latin America . In Egypt, yellow-skinned people were respected for defeating white-skinned people."

Appendix

Table 5: Case Study: CarO vs. baseline reference retrieval in content moderation.

Analogical Reasoning for Language Models

See all 13 papers

ReasoningShield: Safety Detection over Reasoning Traces of Large Reasoning Models
May 22, 2025Changyi Li, Jiayi Wang, Xudong Pan +2Language Model Safety Evaluation LLM Safety Benchmarks
Fudan University · Shanghai Innovation Institute
CHAIRO: Contextual Hierarchical Analogical Induction and Reasoning Optimization for LLMs
Apr 12, 2026Haotian Lu, Yuchen Mou, Bingzhe WuAnalogical Reasoning for Language Models Content Moderation
Shenzhen University · Tsinghua University · National University of Singapore
Tool-MCoT: Tool Augmented Multimodal Chain-of-Thought for Content Safety Moderation
Mar 15, 2026Shutong Zhang, Dylan Zhou, Yinxiao Liu +3Multimodal Content Moderation Content Moderation
Stanford University, USA · Google LLC, USA

May 22, 2025J/K move · Enter open · S save

ReasoningShield: Safety Detection over Reasoning Traces of Large Reasoning Models

Changyi Li, Jiayi Wang, Xudong Pan, Geng Hong+1

Fudan University · Shanghai Innovation Institute

Language Model Safety Evaluation LLM Safety Benchmarks Large Reasoning Models

Open paper AskPDF arXiv

Large Reasoning Models (LRMs) leverage explicit reasoning traces, known as Chain-of-Thought (CoT) reasoning, to decompose complex problems into intermediate steps before deriving final answers. However, these reasoning traces introduce unique safety challenges: harmful content can be embedded in intermediate steps even when final answers appear benign. Our study reveals that existing state-of-the-art moderation tools experience significant performance degradation on CoT moderation, with F1 scores dropping by up to 35.2% compared to traditional answer moderation. To address these challenges, we present ReasoningShield, a comprehensive benchmark and a suite of strong lightweight models for CoT safety moderation. Our benchmark includes 9.2K annotated (Query, CoT) pairs with structured stepwise risk analysis, consisting of a 7K-sample training set and a 2.2K human-annotated test set, spanning 10 risk categories, 3 safety levels, and 8 LRMs with diverse reasoning paradigms. To further demonstrate the utility of the benchmark, we develop lightweight models with 1B/3B parameters using a two-stage training strategy. Our models achieve 91.8% F1 on our benchmark, substantially outperforming leading tools such as LlamaGuard-4 by 35.6% and commercial models such as GPT-4o by 15.8%, while generalizing effectively across diverse reasoning paradigms and unseen scenarios. Resources are publicly available: Project Page: https://cosmosyi.github.io/ReasoningShield/, Code: https://github.com/CosmosYi/ReasoningShield, Models: https://huggingface.co/ReasoningShield/models, and Dataset: https://huggingface.co/datasets/ReasoningShield/ReasoningShield-Dataset.

Abstract

Analogical Reasoning for Language Models

Explore similar work

ReasoningShield: Safety Detection over Reasoning Traces of Large Reasoning Models

CHAIRO: Contextual Hierarchical Analogical Induction and Reasoning Optimization for LLMs

Tool-MCoT: Tool Augmented Multimodal Chain-of-Thought for Content Safety Moderation