PivotAttack: Rethinking the Search Trajectory in Hard-Label Text Attacks via Pivot Words

Yuzhi Liang; Shiliang Xiao; Jingsong Wei; Qiliang Lin; Xia Li

cs.CLMar 11, 2026

PivotAttack: Rethinking the Search Trajectory in Hard-Label Text Attacks via Pivot Words

Authors: Yuzhi Liang, Shiliang Xiao, Jingsong Wei, Qiliang Lin, Xia Li

Organizations: Guangdong University of Foreign Studies, China

Abstract

Existing hard-label text attacks often rely on inefficient "outside-in" strategies that traverse vast search spaces. We propose PivotAttack, a query-efficient "inside-out" framework. It employs a Multi-Armed Bandit algorithm to identify Pivot Sets-combinatorial token groups acting as prediction anchors-and strategically perturbs them to induce label flips. This approach captures inter-word dependencies and minimizes query costs. Extensive experiments across traditional models and Large Language Models demonstrate that PivotAttack consistently outperforms state-of-the-art baselines in both Attack Success Rate and query efficiency.

Figures & tables

Figure 1: The overall workflow of PivotAttack. The framework first identifies Pivot Sets that anchor the model’s prediction, selecting sets with high retention precision and refining them via a multi-armed bandit. It then generates substitutions for the pivot words and selects the variant most similar to the original sentence as the final adversarial example.

Figure 2: Pivot Set Identification on MR

Model	Attack	Yelp		Yahoo		MR		Amazon		SST-2
Model	Attack	ASR $\uparrow$	Pert $\downarrow$	ASR $\uparrow$	Pert $\downarrow$	ASR $\uparrow$	Pert $\downarrow$	ASR $\uparrow$	Pert $\downarrow$	ASR $\uparrow$	Pert $\downarrow$
WordLSTM	PivotAttack	16.8	1.4	42.3	1.5	50.6	5.1	18.6	1.8	37.8	6.1
	LimeAttack	11.9	2.5	39.3	2.9	48.3	5.0	18.2	2.4	36.5	5.8
	TextHacker	14.5	5.9	38.8	4.3	44.9	5.9	18.5	3.5	33.6	6.6
	LeapAttack	10.9	2.6	36.4	3.0	45.2	4.9	15.0	2.2	33.4	5.6
	TextHoaxer	9.5	2.3	34.4	3.8	42.8	4.8	14.4	2.5	29.2	5.5

Table 1: Comprehensive performance comparison (ASR % and Pert %) across all victim models and datasets under a 100-Query Budget.

Figure 3: ASR vs. Query Budget: MR (Qwen2.5-FT)

Dataset	Method	ASR.	Pert.
Yelp	PivotAttack	15.2	1.2
	−Culling	13.8	1.4
	−MAB	13.6	1.4
	−Tighten	14.8	1.3
MR	PivotAttack	49.8	5.0
	−Culling	47.9	5.0

Table 4: Ablation study of the Pivot Set Identification module.

Panel of Figure 4 — Figure 4: Human Evaluation

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Table 6: Original Accuracy of the Victim Models on Classification and NLI Tasks

Panel of Figure 5 — Figure 5: Zero-shot LLM prompts used in hard-label evaluation on different classes. It shows the exact prompts we use during evaluation.

Model	Attack	Yelp		Yahoo		MR		Amazon		SST-2
Model	Attack	ASR $\uparrow$	Pert $\downarrow$	ASR $\uparrow$	Pert $\downarrow$	ASR $\uparrow$	Pert $\downarrow$	ASR $\uparrow$	Pert $\downarrow$	ASR $\uparrow$	Pert $\downarrow$
WordCNN	PivotAttack	12.9	1.6	44.5	2.4	51.5	6.0	19.0	2.1	46.4	6.1
	LimeAttack	11.1	2.3	43.2	2.9	48.8	5.1	17.0	2.4	43.6	5.8
	TextHacker	12.9	3.0	43.0	4.6	48.6	6.1	19.0	2.9	37.2	6.6
	LeapAttack	10.0	2.6	38.1	2.7	46.7	4.9	16.1	2.3	41.7	5.5
	TextHoaxer	9.8	2.6	38.0	2.8	42.3	4.7	16.0	2.5	37.5	5.3

Appendix

Table 7: Comprehensive performance comparison (ASR % and Pert %) across all victim models and datasets under a 100-Query Budget.

Panel of Figure 6 — Figure 6: ASR of different models under different query budgets on MR and SST-2.

Panel of Figure 7 — Figure 7: Example of the Human Evaluation Survey

Adversarial Text Attacks

Adversarial text attacks test how small or meaning-preserving text changes can fool language models, classifiers, and safety systems.

See all 78 papers

OASIS: Optimizing Attacker Sequences for Hard-Label Black-Box Text Attacks
Aug 30, 2026Qian Chen, Shiliang Xiao, Yuzhi LiangAdversarial Attacks Black-Box Adversarial Attacks
School of Information Science and Technology Guangdong University of Foreign Studies, China
NonTextual Target Attack
Oct 3, 2025Xinzhe Huang, Wenjing Hu, Tianhang Zheng +6Adversarial Prompt Generation Adversarial Attacks on LLMs
State Key Laboratory of Blockchain and Data Security, Zhejiang University, Hangzhou 310007, China · Hangzhou High-Tech Zone, Binjiang Institute of Blockchain and Data Security, Hangzhou 310053, China · School of Cyberspace Security, Nanjing University of Science and Technology, Nanjing 210094, China +3
SEP-Attack: A Simple and Effective Paradigm for Transfer-Based Textual Adversarial Attack
May 24, 2026Han Liu, Zhi Xu, Xiaotong Zhang +5Determinantal Point Process Transfer-Based Adversarial Attacks
Dalian University of Technology Dalian, China · Peking University Beijing, China · Macao Polytechnic University Macao, China +1

Aug 30, 2026J/K move · Enter open · S save

OASIS: Optimizing Attacker Sequences for Hard-Label Black-Box Text Attacks

Qian Chen, Shiliang Xiao, Yuzhi Liang

School of Information Science and Technology Guangdong University of Foreign Studies, China

Adversarial Attacks Black-Box Adversarial Attacks Adversarial Text Attacks

Open paper AskPDF arXiv

Different attack methods follow different search trajectories, they succeed on different subsets of samples, whereas existing hard-label black-box text attacks mainly focus on improving individual attackers or manually combining them. We present OASIS, a method for optimizing attacker sequences in hard-label black-box text attacks. OASIS first performs a one-time bi-objective attack chain search over candidate sequences to balance attack success rate and perturbation, and then reuses the selected fixed global chain during attack chain execution. Experiments across multiple datasets, victim models, and large language models show that OASIS consistently outperforms strong standalone baselines and simple manually constructed chains. These results suggest that attacker composition is not merely an implementation choice, but a practical optimization target for improving hard-label black-box text attacks.

Abstract

Adversarial Text Attacks

Explore similar work

OASIS: Optimizing Attacker Sequences for Hard-Label Black-Box Text Attacks

NonTextual Target Attack

SEP-Attack: A Simple and Effective Paradigm for Transfer-Based Textual Adversarial Attack