Multiple-Choice Question Answering

Also known as MCQA

Follow this topic Ask about this topic

Momentum

7 papers in the last four weeks, up 75% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 27

All topics

Penalty-Framed No-Valid-Option MCQA: Analyzing LLM Abstention under Invalid Choices
Oct 6, 2026Jinhyeok Kim, Hye-Young JungLLM Evaluation Multiple-Choice Question Answering
Applied Artificial Intelligence Hanyang University, Republic of Korea · Mathematical Data Science Hanyang University, Republic of Korea
AnyJev Technical Report
Sep 30, 2026Jiamu Zhang, Tianze Yang, Yucheng Shi +6Multiple-Choice Question Answering Language Model Calibration
Nokia, Sunnyvale, CA, USA · Tencent Hunyuan · The Hong Kong Polytechnic University
Likelihood Ranking doesn't Scale Like Prompting in LLMs
Sep 24, 2026Alessandro Bondielli, Lucia Passaro, Davide Bacciu +1LLM Evaluation Language Model Generation Evaluation
CoLingLab, Department of Philology, Literature and Linguistics, University of Pisa · Department of Computer Science, University of Pisa
Reasoning Instructions Can Break Answer Decoding in Vision--Language Models
Sep 24, 2026Zeyan Li, Siyuan Qiu, Jianfeng XuVLM Evaluation Language Model Generation Evaluation
Shanghai Jiao Tong University
Option-Aware Retrieval and Task-Specific VLM Adaptation for Medical VQA
Sep 14, 2026Tristan Kirscher, Niklas C. Koser, Soren PirkVLM Adaptation Multiple-Choice Question Answering
ICube, Institut Strauss · ICube Laboratory, CNRS UMR 7357, University of Strasbourg, Strasbourg, France · CLCC Institut Strauss, Strasbourg, France +2
Turkish MMLU Pro: Traceable Option Augmentation and Its Validity Limits in Turkish Multiple-Choice Evaluation
Sep 14, 2026M. Ali BayramMultilingual Language Model Evaluation LLM Evaluation
Department of Computer Engineering, Yıldız Technical University İstanbul, Turkey
EAR: Entity-Aware Partitioning Approach for Retrieval-Augmented Generation Development
Sep 14, 2026Cenab Batu Bora, Oylum Alatl\i, Sebnem Bora +1Document Chunking Retrieval-Augmented Generation
Georgia Institute of Technology, Atlanta, GA, USA · Ege University, Izmir, Turkey
When Models Defer to Wrong Answers: A Robustness Audit of Source-Attributed Cues in Multiple-Choice QA
Sep 8, 2026Manikandan Ravikiran, Siddharth VohraMultiple-Choice Question Answering LLM Reliability
Indian Institute of Technology Mandi, India · Carnegie Mellon University Amazon Web Services AI Native Pittsburgh, PA, USA
StateSwap: Probing Support-Elimination Hidden States in Multiple-Choice Questions
Sep 1, 2026Chao Gao, Haijiang Liu, Qiyuan Li +3Prompt Sensitivity Multiple-Choice Question Answering
Department of Computer Science, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands · School of Computer Science and Technology, Wuhan University of Science and Technology, Wuhan, China · Hubei Province Key Laboratory of Intelligent Information Processing and Real-time Industrial System, Wuhan University of Science and Technology, Wuhan, China
Check The Scoreboard: An Analysis of Scoring Schemes on Multiple-Choice Evaluation
Aug 30, 2026Nishant Balepur, Paiheng Xu, Wei Ai +3LLM Evaluation Multiple-Choice Question Answering
University of Maryland · New York University · Nanyang Technological University
From Atomic Evidence to Logical Composition: Structured Compositional Reasoning over Compound Answer Options
Aug 13, 2026Obed Junias, Maria Leonor PachecoCompositional Reasoning Multiple-Choice Question Answering
University of Colorado Boulder
Accuracy and Order Sensitivity Diverge Under Label-Free Strategies
Aug 12, 2026Karl Hanna, Chen FengLLM Evaluation Multiple-Choice Question Answering
Queen’s University Belfast
Isotropy Cliffs: The Geometric Signature of Decision-Making in Large Language Models
Aug 1, 2026Okan S. Coskun, Florian Rottach, Carsten Eickhoff +1Multiple-Choice Question Answering LLM Decision-Making
University of Tubingen · The University of Texas at Austin
Language-Routed RAG and Direct Option Scoring for Multilingual Financial QA: DS@GT at FinMMEval
Jul 24, 2026Justice Ayela, Kabir SahniMultilingual Language Model Evaluation Retrieval-Augmented Generation
Georgia Institute of Technology, North Ave NW, Atlanta, GA 30332
Overview of FinMMEval 2026 Task 1: Multilingual Financial Multiple-Choice Question Answering
Jul 22, 2026Zhuohan Xie, Yuyang Dai, Rania Elbadry +18Multilingual Language Model Evaluation Multiple-Choice Question Answering
Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates · INSAIT, Sofia University “St. Kliment Ohridski”, Sofia, Bulgaria · University of Arizona, Tucson, United States +6
Punching Above Their Weight: Classification-Head Fine-Tuning of Tiny Language Models (TLMs) for Verifiable Multiple-Choice Tasks
Jul 4, 2026Bhavesh Sood, Jaromir SavelkaFine-Tuning Multiple-Choice Question Answering
Carnegie Mellon University
Persona Non Grata: LLM Persona-Driven Generations in MCQA are Unstable in Distinct Dimensions
Jul 1, 2026César Guerra-Solano, Xiang Lorraine LiLLM Evaluation Language Model Generation Evaluation
Department of Computer Science, University of Pittsburgh
Are We Evaluating Knowledge or Phrasing? Mitigating MCQA Sensitivity with ParaEval
Jun 9, 2026João Maria Janeiro, Mathurin Videau, Andrea Caciolai +3LLM Evaluation Multiple-Choice Question Answering
FAIR at Meta · Sorbonne Université, CNRS, ISIR, F-75005 Paris, France · Criteo AI Lab, Paris, France
Second Guess: Detecting Uncertainty Through Abstention and Answer Stability in Small Language Models
May 25, 2026Ashwath Vaithinathan Aravindan, Mayank KejriwalConfidence Estimation in Language Models Multiple-Choice Question Answering
University of Southern California Los Angeles, 90007, California, United States of America · Information Sciences Institute 4676 Admiralty Way #1001, Los Angeles, 90292, California, United States of America
Code-Guided Reasoning for Small Language Models: Evaluating Executable MCQA Scaffolds
May 12, 2026Prateek Biswas, Dhaval Patel, Vedant Khandelwal +2Multiple-Choice Question Answering Small Language Models
1IBM · 2Artificial Intelligence Institute at University of South Carolina
ReasonLab: A Controlled and Auditable Evaluation of Prompting Techniques for Multiple-Choice QA
May 7, 2026Inder Preet, Shuxin Lin, Dhaval PatelLLM Evaluation Language Model Generation Evaluation
IBM, Dublin · IBM, New York
Evaluating Large Language Models on Computer Science University Exams in Data Structures
Apr 25, 2026Edan Gabay, Yael Maoz, Jonathan Stahl +7LLM Evaluation Computer Science Education
Blavatnik School of Computer Science and AI
More Thinking, More Bias: Length-Driven Position Bias in Reasoning Models
Apr 21, 2026Xiao WangMultiple-Choice Question Answering CoT Reasoning
Pushing the Boundaries of Multiple Choice Evaluation to One Hundred Options
Apr 16, 2026Nahyun Lee, Guijin SonLLM Evaluation Multiple-Choice Question Answering
Chung-Ang University · Seoul National University
Corpus Prevalence of Multiple-Choice Question Options
Feb 19, 2026Leonidas Zotos, Hedderik van Rijn, Malvina NissimMultiple-Choice Question Answering Generative AI in Education
Center for Language and Cognition, University of Groningen, Oude Kijk in ’t Jatstraat 26, Groningen, 9712 EK, The Netherlands · Department of Experimental Psychology, University of Groningen, Grote Kruisstraat 2/1, Groningen, 9712 TS, The Netherlands
Robustness assessment of large audio language models in multiple-choice evaluation
Oct 6, 2025Fernando López, Santosh Kesiraju, Jordi LuqueAudio-Language Model Evaluation Audio QA
Scientific Research, Telef´onica Innovaci´on Digital, Spain · Brno University of Technology, Czech Republic
Listening to the Wise Few: Query-Key Alignment Unlocks Latent Correct Answers in Large Language Models
Oct 3, 2024Eduard Tulchinskii, Kristian Kuznetsov, Laida Kushnareva +5Self-Attention Multiple-Choice Question Answering
Skolkovo Institute of Science and Technology · AI Foundation and Algorithm Lab · Moscow Institute of Physics and Technology +2

Oct 6, 2026, cs.CLJ/K move · Enter open · S save

Penalty-Framed No-Valid-Option MCQA: Analyzing LLM Abstention under Invalid Choices

Jinhyeok Kim, Hye-Young Jung

Applied Artificial Intelligence Hanyang University, Republic of Korea · Mathematical Data Science Hanyang University, Republic of Korea

LLM Evaluation Multiple-Choice Question Answering LLM Reliability LLM Abstention

Open paper AskPDF arXiv

Multiple-choice question answering (MCQA) is commonly used to evaluate large language models under the assumption that one of the provided options is correct, typically using answer-selection accuracy. However, in real deployments, users or retrieval systems may provide invalid option sets in which none of the listed choices is correct, and selecting one of them may incur downstream cost. We study this setting as penalty-framed no-valid-option MCQA. Using the mathematics subset of MMLU-Pro, we remove the labeled correct option, allow models to either choose a remaining option or output ABSTAIN, and penalize invalid forced-choice responses. We further introduce correct-conditioned analysis, evaluating abstention only on instances that the model originally answered correctly. Experiments show that high MCQA accuracy does not fully guarantee abstention reliability: even under explicit no-valid-option-aware instructions and penalty-based scoring, models still produce invalid forced-choice responses for a subset of originally correct instances. These results show that penalty-framed no-valid-option MCQA reveals an aspect of model reliability not captured by standard answer-selection accuracy.

Momentum

Related topics

Latest papers 27