Shortcut Learning in Language Models

Latest papers

11 papers2 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Auditing Tool Traces in Agentic LLMs: Source Localization and Counterfactual Dependence Testing
Oct 18, 2026Junhao Liu, Tianxiao Li, Yongxian Wu +4LLM Auditing AI Agent Auditing
University of California, Irvine · NEC Labs America
DeShortcut-Align: Decoupling Spurious Shortcuts for Robust Safety Alignment in Large Reasoning Models
Sep 28, 2026Qirui Liu, Yichen Sun, Yan Wang +4Shortcut Learning in Language Models LLM Safety Alignment
The State Key Laboratory of Blockchain and Data Security, Zhejiang University · Ant Group · Chongqing Ant Consumer Finance Co., Ltd
When Refusal Looks Safe: The Refusal-Cue Shortcut in Safety Guard Models
Aug 4, 2026Yu Feng, Chunting Zang, Chen Shen +4Shortcut Learning in Language Models Language Model Safety Evaluation
The University of Sydney · Alibaba Group · Jilin University +1
To Reason or to Fabricate: Reasoning Without Shortcuts via Hint-Anchored Pairwise Aggregation
Jun 28, 2026Jiuheng Lin, Chen Zhang, Yansong FengCoT Faithfulness Reward Modeling
Wangxuan Institute of Computer Technology, Peking University
What's in a Name? Morphological Shortcuts by LLMs in Pharmacology
Jun 4, 2026Kaijie Mo, Thomas Yang, Chantal Shaib +6Shortcut Learning in Language Models Hallucination in Language Models
The University of Texas at Austin · Northeastern University · MD Anderson Cancer Center
Caliper: Probing Lexical Anchors versus Causal Structure in LLMs
Jun 3, 2026Zhenyu Yu, Shuigeng ZhouShortcut Learning in Language Models Causal Reasoning in Language Models
Fudan University
The Paradox of Outcome Optimization: A Causal Information-Theoretic Bound on Reasoning Shortcuts in LLMs
May 30, 2026Zihan Chen, Yiming Zhang, Wenxiang Geng +2Shortcut Learning in Language Models Process Reward Models
1HFIPS, Chinese Academy of Sciences · University of Science and Technology of China
Calibration vs Decision Making: Revisiting the Reliability Paradox in Unlearned Language Models
May 20, 2026Divyaksh Shukla, Ashutosh ModiShortcut Learning in Language Models Confidence Estimation in Language Models
Indian Institute of Technology Kanpur (IIT Kanpur)
LLMs taking shortcuts in test generation: A study with SAP HANA and LevelDB
Apr 15, 2026Vekil Bekmyradov, Noah C. Pütz, Thomas Bartz-BeielsteinShortcut Learning in Language Models LLM Evaluation
THK-AI Research Cluster, TH Köln
Mitigating Shortcut Reasoning in Language Models: A Gradient-Aware Training Approach
Mar 21, 2026Hongyu Cao, Kunpeng Liu, Dongjie Wang +1Shortcut Learning in Language Models LLM Reasoning
Arizona State University · Clemson University · University of Kansas
Shattering the Shortcut: A Topology-Regularized Benchmark for Multi-hop Medical Reasoning in LLMs
Mar 12, 2026Xing Zi, Xinying Zhou, Jinghao Xiao +2Shortcut Learning in Language Models Multi-Hop QA
University of Technology Sydney, Australia

Oct 18, 2026J/K move · Enter open · S save

Auditing Tool Traces in Agentic LLMs: Source Localization and Counterfactual Dependence Testing

Junhao Liu, Tianxiao Li, Yongxian Wu, Ziheng Duan+3

University of California, Irvine · NEC Labs America

LLM Auditing AI Agent Auditing Shortcut Learning in Language Models Scientific Reasoning in Language Models Failure Attribution for LLM Agents

Open paper AskPDF arXiv

Agentic LLMs increasingly support scientific prediction by grounding decisions in external evidence. In biology, they promise accurate, interpretable predictions for important problems by integrating complex experimental and computational evidence. However, serialized tool results can contain hidden label shortcuts, such as dataset identifiers or template artifacts, that create apparent success without biological reasoning. To address this risk, we introduce a two-stage audit. Stage 1, Per-Tool TF-IDF Decomposition (PT2D), localizes label-predictive patterns to individual tool results while avoiding signal dilution in the concatenated tool trace. Stage 2, Token-Class Redaction, counterfactually tests whether the frozen LLM depends on the localized shortcut. We apply this audit to TCR-pMHC binding, where tool augmentation substantially outperforms a no-evidence baseline. At face value, this suggests that the LLM has learned to use scientific evidence. The audit overturns this interpretation: PT2D localizes the strongest signal to identifiers in one tool result, and redacting them reduces performance to chance. These findings show how hidden label shortcuts can masquerade as scientific reasoning or discovery, motivating source-level audits before benchmark gains are trusted.