Data Poisoning Attacks

Follow this topic Ask about this topic

Momentum

12 papers in the last four weeks, up 50% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 93

All topics

Alignment Tampering: How Reinforcement Learning from Human Feedback Is Exploited to Optimize Misaligned Biases
May 26, 2026Dongyoon Hahm, Dylan Hadfield-Menell, Kimin LeeLLM Alignment Data Poisoning Attacks
1KAIST · 2MIT
Building an Adversarial Malware Dataset by Family and Type: Generation, Evasion, and Poisoning Evaluation
May 25, 2026David Košťál, Martin JurečekAdversarial Attacks Malware Classification
Department of Information Security, Faculty of Information Technology, Czech Technical University in Prague
Security in the Fine-Tuning Lifecycle of Large Language Models: Threats, Defenses,Evaluation, and Future Directions
May 24, 2026Wenjuan Li, Yitao Liu, Runze Chen +1LLM Backdoor Attacks LLM Fine-Tuning
Hangzhou Normal University, No. 2318, Yuhangtang Rd, Hangzhou, 311121, Zhejiang, China · Zhejiang University, No. 866 Yuhangtang Rd, Hangzhou, 2026 310058, Zhejiang, China · China Mobile (Zhejiang) Innovation Research Institute Co., Ltd., No. 77 Gaoxin 7th Rd, China Mobile Zhejiang Information and Communication Industrial Park, Hangzhou, 311200, Zhejiang, China +1
PoisonForge: Task-Level Targeted Poisoning Benchmark for Instruction-Tuned LLMs
May 22, 2026Luze Sun, Anshuman Suri, Harsh Chaudhari +2LLM Safety Benchmarks LLM Backdoor Attacks
Department of Computer Science Northeastern University · DatologyAI
When Stronger Triggers Backfire: A High-Dimensional Theory of Backdoor Attacks
May 21, 2026Donald Flynn, Hadas Yaron Goldhirsh, Jonathan P. Keating +1Backdoor Attacks Statistical Learning Theory
1Mathematical Institute, University of Oxford · School of Mathematical Science, Tel Aviv University · School of Mathematical Science and Computer Science, Tel Aviv University
Token by Token, Compromised: Backdoor Vulnerabilities in Unified Autoregressive Models
May 19, 2026Tobias Braun, Jonas Henry Grebe, Hossein Shakibania +2Unified Multimodal Models Backdoor Attacks
TU Darmstadt & hessian.AI, Germany
Be Kind, Rewrite: Benign Projections via Rewriting Defend Against LLM Data Poisoning Attacks
May 18, 2026John T. Halloran, Noopur S. BhattLLM Backdoor Attacks Training Data Curation
Leidos · University of Pennsylvania
SoK: A Comprehensive Analysis of the Current Status of Neural Tangent Generalization Attacks with Research Directions
May 12, 2026Thushari Hapuarachchi, Kaiqi XiongAdversarial Attacks Adversarial Robustness
University of South Florida Tampa, USA
Oracle Poisoning: Corrupting Knowledge Graphs to Weaponise AI Agent Reasoning
May 10, 2026Ben Kereopa-Yorke, Guillermo Diaz, Holly Wright +3AI Agent Security LLM Agent Security
1Microsoft · 2UNSW Canberra · 3SAP +1
ATAAT: Adaptive Threat-Aware Adversarial Tuning Framework against Backdoor Attacks on Vision-Language-Action Models
May 9, 2026Kewei Chen, Yayu Long, Shuai Li +1Adversarial Attacks Adversarial Attacks on VLA Models
Chongqing Institute of Green and Intelligent Technology, Chinese Academy of Sciences · Chongqing School, University of Chinese Academy of Sciences · Faculty of Information Technology and Electrical Engineering, University of Oulu, Finland
Narrow Secret Loyalty Dodges Black-Box Audits
May 7, 2026Alfie Lamerton, Fabien RogerData Poisoning Attacks Black-Box Model Auditing
Formation Research
Gray-Box Poisoning of Continuous Malware Ingestion Pipelines
May 6, 2026Jan Dolejš, Martin Jureček, Róbert LórenczAdversarial Robustness Malware Analysis
Faculty of Information Technology, Czech Technical University in Prague, Thákurova 9, Prague, 16000, Czech Republic
Efficient Preference Poisoning Attack on Offline RLHF
May 4, 2026Chenye Yang, Weiyu Xu, Lifeng LaiDirect Preference Optimization Data Poisoning Attacks
Department of Electrical and Computer Engineering, University of California, Davis, Davis, CA, USA · Department of Electrical and Computer Engineering, University of Iowa, Iowa City, IA, USA.
Repurposing and Evaluating the (In)Feasibility of Dataset Poisoning enabled Watermarking for Contrastive Learning
May 3, 2026Zhiyang Dai, Yansong Gao, Boyu Kuang +5Contrastive Learning Dataset Watermarking
School of Cyber Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China · University of Western Australia, Perth, Australia · College of Electronics and Information Engineering, Shenzhen University, Shenzhen, China +3
Checkerboard: A Simple, Effective, Efficient and Learning-free Clean Label Backdoor Attack with Low Poisoning Budget
May 2, 2026Yi Yang, Jinyang Huang, Binbin Liu +5Backdoor Attacks Data Poisoning Attacks
School of Computer Science and Information Engineering Hefei University of Technology Hefei, Anhui, China · School of Information Science and Technology University of Science and Technology of China Hefei, Anhui, China · Faculty of Business Data Science Kansai University Japan +3
AdaBFL: Multi-Layer Defensive Adaptive Aggregation for Bzantine-Robust Federated Learning
Apr 30, 2026Zehui Tang, Yuchen Liu, Feihu HuangFederated Learning Aggregation Byzantine-Robust Federated Learning
College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing, China · MIIT Key Laboratory of Pattern Analysis and Machine Intelligence, Nanjing, China.
Toward Polymorphic Backdoor against Semantic Communication via Intensity-Based Poisoning
Apr 25, 2026Xiao Yang, Yuni Lai, Gaolei Li +4Backdoor Attacks Adversarial Robustness
School of Computer Science, Shanghai Jiao Tong University, and Shanghai Key Laboratory of Integrated Administration Technologies for Information Security, Shanghai, China · Department of Computing, Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong · Department of Electrical and Computer Engineering and Frost Institute for Data Science and Computing, University of Miami, Coral Gables, Florida 33146, USA
Sharpness-Aware Poisoning: Enhancing Transferability of Injective Attacks on Recommender Systems
Apr 24, 2026Junsong Xie, Yonghui Yang, Pengyang Shao +1Sharpness-Aware Minimization Adversarial Attacks
Key Laboratory of Knowledge Engineering with Big Data, Hefei University of Technology, Hefei 230009, China · Intelligent Interconnected Systems Laboratory of Anhui Province, Hefei University of Technology, Hefei 230009, China · Laboratory of NExT++, National University of Singapore, Singapore
PermaFrost-Attack: Stealth Pretraining Seeding(SPS) for planting Logic Landmines During LLM Training
Apr 23, 2026Harsh Kumar, Rahul Maity, Tanmay Joshi +4Language Model Pretraining LLM Backdoor Attacks
1Manipal University Jaipur · 2NIT, Karnataka · 3Pragya Lab, BITS Pilani Goa, India +2
Stealthy Backdoor Attacks against LLMs Based on Natural Style Triggers
Apr 23, 2026Jiali Wei, Ming Fan, Guoheng Sun +3LLM Backdoor Attacks Backdoor Attacks
School of Cyber Science and Engineering, Xi’an Jiaotong University, Xi’an 710049, China · Ministry of Education Key Lab for Intelligent Networks and Network Security, Xi’an Jiaotong University, Xi’an 710049, China
CSC: Turning the Adversary's Poison against Itself
Apr 23, 2026Yuchen Shi, Xin Guo, Huajie Chen +3Backdoor Attacks Backdoor Defense
Faculty of Data Science, City University of Macau, Taipa, Macau. · Faculty of Engineering and Information Technology, University of Technology Sydney, New South Wales, Australia.
Detecting Data Poisoning in Code Generation LLMs via Black-Box, Vulnerability-Oriented Scanning
Mar 17, 2026Shenao Yan, Shan Jin, Shimaa Ahmed +4LLM Auditing Backdoor Detection
University of Connecticut · Visa Research
Are Targeted Data Poisoning Attacks as Effective as We Think?
Sep 8, 2025William Xu, Chenyu Zhang, Yihan Wang +5Data Poisoning Attacks
Waabi AI · University of Waterloo · Carnegie Mellon University +3
Data Security in Large Language Models: Risks, Defense, and Directions
Aug 4, 2025Kang Chen, Xiuze Zhou, Yuanhui Yu +4Adversarial Attacks on LLMs LLM Security
School of Computer Engineering, Jimei University, Xiamen, 361021, China · The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, 511453, China · School of Informatics, Xiamen University, Xiamen, 361102, China +1
Tuning without Peeking: Provable Generalization Bounds and Robust LLM Post-Training
Jul 2, 2025Ismail Labiad, Mathurin Videau, Matthieu Kowalski +4Privacy-Preserving Language Models Evolutionary Optimization
Meta FAIR · Université Paris-Saclay · LISN, Inria, CNRS +2
Tight Stability Bounds for Robust Distributed Learning: Byzantine Failures Hurt Generalization More than Data Poisoning
Jun 22, 2025Thomas Boudou, Batiste Le Bars, Nirupam Gupta +1Byzantine-Robust Federated Learning Algorithmic Stability
PreMeDICaL team, Inria, Idesp, Inserm, Université de Montpellier, Montpellier, France · Univ. Lille, Inria, CNRS, Centrale Lille, UMR 9189, CRIStAL, F-59000 Lille, France · Department of Computer Science, University of Copenhagen, Copenhagen, Danemark
Fox in the Henhouse: Supply-Chain Backdoor Attacks Against Reinforcement Learning
May 26, 2025Shijie Liu, Andrew C. Cullen, Paul Montague +2Adversarial Attacks Backdoor Attacks
School of Computing and Information Systems, University of Melbourne, Parkville, Australia · DST Group, Adelaide
How to Backdoor Image Knowledge Distillation
Apr 30, 2025Qian Ma, Chen Wu, Prasenjit Mitra +1Adversarial Attacks Data Poisoning Attacks
The Pennsylvania State University State College, USA · Carnegie Mellon University Africa Kigali, Rwanda
Trust Under Siege: Label Spoofing Attacks against Machine Learning for Android Malware Detection
Mar 14, 2025Tianwei Lan, Luca Demetrio, Farid Nait-Abdesselam +2Android Malware Detection Malware Classification
1Université Paris Cité, France · University of Genova, Italy · 3INRIA, France +1
Approaching the Harm of Gradient Attacks While Only Flipping Labels
Feb 28, 2025Abdessamad El-Kabid, El-Mahdi El-MhamdiFederated Learning Aggregation Adversarial Attacks
École Polytechnique

May 26, 2026, cs.AIJ/K move · Enter open · S save

Alignment Tampering: How Reinforcement Learning from Human Feedback Is Exploited to Optimize Misaligned Biases

Dongyoon Hahm, Dylan Hadfield-Menell, Kimin Lee

1KAIST · 2MIT

LLM Alignment Data Poisoning Attacks Value Alignment RL from Human Feedback

Open paper AskPDF arXiv

Reinforcement Learning from Human Feedback (RLHF) is the standard method to align Large Language Models (LLMs) with human preferences. In this work, we introduce alignment tampering, a potential vulnerability where the LLM undergoing alignment influences the preference dataset, causing RLHF to amplify undesired behaviors. This arises from core limitations of RLHF: (1) preference datasets are constructed from the LLM's own outputs, allowing it to influence them, and (2) pairwise comparisons only indicate which response is better, not why. These limitations can be exploited to cause alignment tampering. For example, if an LLM generates biased responses with higher quality, annotators will prefer them based on quality. However, preference labels do not distinguish quality from bias, and the reward model inherits this limitation. Optimizing such rewards through reinforcement learning or best-of-N sampling can amplify misaligned biases. Our experiments demonstrate amplification across diverse biases: from keyword bias to propaganda (e.g., sexism), brand promotion, and instrumental goal-seeking. Mitigation remains challenging, as existing techniques for robust RLHF fail to fully resolve alignment tampering without sacrificing response quality. These findings reveal structural vulnerabilities of current RLHF and emphasize the need to prevent this vulnerability. Project page: https://alignment-tampering.github.io/

Momentum

Related topics

Latest papers 93