Input Order Sensitivity in Language Models

Latest papers

14 papers5 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Joint or Per-Option? Compact Encoders for Closed-Set Decisions from 17 Million to 1 Billion Parameters
Oct 22, 2026Tardelli Ronan Coelho StekelInput Order Sensitivity in Language Models LLM Decision-Making
Federal Institute of São Paulo (IFSP), São Paulo, Brazil
Where Prefill Expert Reduction Hurts MoE Models: A Few Prompt Tokens Carry Most of the Short-Answer F1 Loss
Oct 15, 2026Zhiyuan Wang, Han Wang, Shuo ZhangInput Order Sensitivity in Language Models Mixture-of-Experts Inference
Independent Researcher
One Bias Score, Two Factors: How Mitigations Affect Option Bias in Multiple-Choice Evaluation
Oct 14, 2026Hyunjong Ok, Minjae Park, Jaeho LeeInput Order Sensitivity in Language Models Language Model Bias Evaluation
Pohang University of Science and Technology (POSTECH)
Consider, Then Choose: Order-Invariant Decisions in System One Models
Oct 13, 2026Jianguo Huang, Yao Long Teng, Yanchen Deng +1Input Order Sensitivity in Language Models Multiple-Choice Question Answering
Nanyang Technological University, Singapore
Input-Blind Controls Produce Substantial Oracle Headroom for Layer Programs in Multiple-Choice Evaluation
Oct 7, 2026Yibei Guo, Rui LiuLayer Skipping in Language Model Decoding Input Order Sensitivity in Language Models
College of Aeronautics and Engineering, Kent State University
A, B, C, or Stable? Accuracy and Order Sensitivity Diverge in LLM Multiple-Choice Evaluation
Aug 12, 2026Karl Hanna, Chen FengLLM Evaluation Multiple-Choice Question Answering
Queen’s University Belfast
Are LLMs Positionally Consistent Ordinal Classifiers? A Systematic Evaluation
Aug 9, 2026Yu Wang, Zhe Zhou, Menglin Liu +1Language Model Bias Evaluation Classification
Cornell University · University of Washington · The Chinese University of Hong Kong, Shenzhen +1
The yes-no bias of large language models reflects answer order and wording, not shifts in moral judgment
Jul 6, 2026Haonan HuangMoral Reasoning in Language Models Framing Effects in Language Models
Princeton University, Princeton, NJ 08540, USA
Same Evidence, Different Answer: Auditing Order Sensitivity in Multimodal Large Language Models
Jun 24, 2026Akshay Paruchuri, Sanmi Koyejo, Ehsan AdeliMultimodal Robustness LLM Auditing
Stanford University
Language Models as Interfaces, Not Oracles: A Hybrid LLM-ML System for Pediatric Appendicitis
Jun 17, 2026Soheyl Bateni, Maryam AbdolaliMedical Diagnosis Clinical Decision Support
K. N. Toosi University of Technology, Tehran, Iran
Decoding in Order-Agnostic Language Models: Chain-Rule Deviation and Uniform Spreading
May 31, 2026Lin YaoLanguage Model Decoding Confidence Estimation in Language Models
School of Computer Science, Shanghai Jiao Tong University, Shanghai, 200240, China · Zhongguancun Academy, Beijing, 100097, China
Towards Order Fairness: Mitigating LLMs Order Sensitivity through Dual Group Advantage Optimization
May 12, 2026Xu Chu, Guanyu Wang, Zhijie Tan +4Language Model Robustness Input Order Sensitivity in Language Models
School of Software and Microelectronics, Peking University, Beijing, China
How Order-Sensitive Are LLMs? OrderProbe for Deterministic Structural Reconstruction
Jan 13, 2026Zhaolu Kang, Yingjie He, Kehan Jiang +18Input Order Sensitivity in Language Models LLM Evaluation
Hunyuan, Tencent · Peking University · Tsinghua University +8
Hearing the Order: Investigating Position Bias in Large Audio-Language Models
Oct 1, 2025Yu-Xiang Lin, Chen-An Li, Sheng-Lun Wei +3Audio-Language Model Evaluation Audio-Language Models
National Taiwan University

Oct 22, 2026J/K move · Enter open · S save

Joint or Per-Option? Compact Encoders for Closed-Set Decisions from 17 Million to 1 Billion Parameters

Tardelli Ronan Coelho Stekel

Federal Institute of São Paulo (IFSP), São Paulo, Brazil

Input Order Sensitivity in Language Models LLM Decision-Making Efficient Language Model Inference LLM Reranking LLM Evaluation

Open paper AskPDF arXiv

Many language-model pipelines end in a closed-set decision, such as selecting a tool. Prompting a large language model for it is slow and order-sensitive. We ask how small an encoder for such decisions can be and whether it should read the options jointly or separately. In a three-seed study at 32 million parameters, per-option scoring beats joint scoring on a held-out sample of a 37-benchmark decision suite by 5.8 points of chance-corrected skill at matched optimizer steps and by 2.8 at approximately matched training compute; two external benchmarks agree, though not significantly on their unseen tasks. Crossing reader with initialisation shows that the advantage requires a reranker initialisation, whose query-candidate pair input matches per-option reading; from the raw encoder the two readers are indistinguishable. Per-option scoring needs 5 to 13 times more encoder passes: at most twice the median latency, but more than four times the mean. Of six released joint encoders (17 million to 1 billion parameters), the 68- and 150-million-parameter models exceed a 435-million-parameter DeBERTa-v3 zero-shot classifier by 4.9 and 5.9 accuracy points on 209 overlap-audited System1-Arena tasks and are at least on par with it on the 145 tasks unseen in training. The smallest answers a median request in 14 milliseconds on 16 processor threads, against 728 for the classifier. Reordering options changes 12 to 23% of the joint models' answers (per-option scoring: none). Up to 68 million parameters, they lead their size range on a public leaderboard's private, mainly in-distribution tests.