Shortcut Learning in Language Models

Latest papers

10 papers1 paper in the last four weeks, with none the four weeks before. 0.0% of all new papers.

DeShortcut-Align: Decoupling Spurious Shortcuts for Robust Safety Alignment in Large Reasoning Models
Sep 28, 2026Qirui Liu, Yichen Sun, Yan Wang +4Shortcut Learning in Language Models LLM Safety Alignment
The State Key Laboratory of Blockchain and Data Security, Zhejiang University · Ant Group · Chongqing Ant Consumer Finance Co., Ltd
When Refusal Looks Safe: The Refusal-Cue Shortcut in Safety Guard Models
Aug 4, 2026Yu Feng, Chunting Zang, Chen Shen +4Shortcut Learning in Language Models Language Model Safety Evaluation
The University of Sydney · Alibaba Group · Jilin University +1
To Reason or to Fabricate: Reasoning Without Shortcuts via Hint-Anchored Pairwise Aggregation
Jun 28, 2026Jiuheng Lin, Chen Zhang, Yansong FengCoT Faithfulness Reward Modeling
Wangxuan Institute of Computer Technology, Peking University
What's in a Name? Morphological Shortcuts by LLMs in Pharmacology
Jun 4, 2026Kaijie Mo, Thomas Yang, Chantal Shaib +6Shortcut Learning in Language Models Hallucination in Language Models
The University of Texas at Austin · Northeastern University · MD Anderson Cancer Center
Caliper: Probing Lexical Anchors versus Causal Structure in LLMs
Jun 3, 2026Zhenyu Yu, Shuigeng ZhouShortcut Learning in Language Models Causal Reasoning in Language Models
Fudan University
The Paradox of Outcome Optimization: A Causal Information-Theoretic Bound on Reasoning Shortcuts in LLMs
May 30, 2026Zihan Chen, Yiming Zhang, Wenxiang Geng +2Shortcut Learning in Language Models Process Reward Models
1HFIPS, Chinese Academy of Sciences · University of Science and Technology of China
Calibration vs Decision Making: Revisiting the Reliability Paradox in Unlearned Language Models
May 20, 2026Divyaksh Shukla, Ashutosh ModiShortcut Learning in Language Models Confidence Estimation in Language Models
Indian Institute of Technology Kanpur (IIT Kanpur)
LLMs taking shortcuts in test generation: A study with SAP HANA and LevelDB
Apr 15, 2026Vekil Bekmyradov, Noah C. Pütz, Thomas Bartz-BeielsteinShortcut Learning in Language Models LLM Evaluation
THK-AI Research Cluster, TH Köln
Mitigating Shortcut Reasoning in Language Models: A Gradient-Aware Training Approach
Mar 21, 2026Hongyu Cao, Kunpeng Liu, Dongjie Wang +1Shortcut Learning in Language Models LLM Reasoning
Arizona State University · Clemson University · University of Kansas
Shattering the Shortcut: A Topology-Regularized Benchmark for Multi-hop Medical Reasoning in LLMs
Mar 12, 2026Xing Zi, Xinying Zhou, Jinghao Xiao +2Shortcut Learning in Language Models Multi-Hop QA
University of Technology Sydney, Australia

Sep 28, 2026J/K move · Enter open · S save

DeShortcut-Align: Decoupling Spurious Shortcuts for Robust Safety Alignment in Large Reasoning Models

Qirui Liu, Yichen Sun, Yan Wang, Zhixuan Chu+3

The State Key Laboratory of Blockchain and Data Security, Zhejiang University · Ant Group · Chongqing Ant Consumer Finance Co., Ltd

Shortcut Learning in Language Models LLM Safety Alignment Adversarial Robustness of Language Models Over-Refusal

Open paper AskPDF arXiv

Safety alignment of large reasoning models (LRMs) via supervised fine-tuning (SFT) and reinforcement learning (RL) often yields near-perfect safety scores, yet this apparent success comes at the cost of severe over-refusal and degraded general capabilities. Through systematic empirical analysis, we find that these failures are closely associated with the learning of spurious shortcuts rather than robust intent-sensitive safety evaluation. Specifically, we identify two dominant shortcuts: formatting shortcuts, where refusal behaviors are overly bound to structural prompt templates that frequently appear in safety alignment corpora; and lexical shortcuts, where sensitive keywords reflexively trigger refusals on benign queries. To mitigate reliance on these shortcuts, we propose DeShortcut-Align, a shortcut-decoupling alignment framework that reduces dependence on superficial cues. DeShortcut-Align operates across three coordinated stages: (1) Refusal Sensitivity Attribution, which masks input tokens to quantify their impact on the final refusal response distribution; (2) Attribution-Guided Contrastive Augmentation, which constructs benign contrastive samples using high-sensitivity tokens to mitigate lexical shortcuts; and (3) Counterfactual Consistency Regularization, which constructs template-ablated states via attention blinding to enforce decision consistency across SFT and RL, mitigating formatting shortcut dependence. Experiments on 7B and 14B models demonstrate that DeShortcut-Align significantly improves robustness against template-stripping bypass attacks (reducing performance drops by up to 72%), substantially reduces over-refusal by over 58%, and better preserves general-purpose reasoning capabilities, thereby mitigating the alignment tax commonly observed in safety training.