Multiple-Choice Question Answering

Also known as MCQA

Multiple-choice question answering (MCQA) studies how models use knowledge, reasoning, and evidence to choose among candidate answers, across settings such as MMLU, HellaSwag, ScienceQA, and medical or multilingual exams. Common approaches include chain-of-thought prompting, likelihood and option scoring, retrieval-augmented generation, fine-tuning with classification heads, and methods that compare or filter options. Researchers also test efficiency and internal decision processes, including compact decision heads, executable reasoning scaffolds, and interventions on model representations. A central tension is that accuracy can hide sensitivity to option order, wording, popularity, and candidate-set size, as well as failures to abstain when choices are invalid.

Latest papers

43 papers9 papers in the last four weeks, up 13% on the four weeks before. 0.1% of all new papers.

Second Guess: Detecting Uncertainty Through Abstention and Answer Stability in Small Language Models
May 25, 2026Ashwath Vaithinathan Aravindan, Mayank KejriwalConfidence Estimation in Language Models Multiple-Choice Question Answering
University of Southern California Los Angeles, 90007, California, United States of America · Information Sciences Institute 4676 Admiralty Way #1001, Los Angeles, 90292, California, United States of America
Where Cross-Question Structure in LLM Errors Comes From: Shared Seeds, Shared Prompts, and Answer-Letter Preference
May 18, 2026Izhar AliCorrelated Errors in Language Models Multiple-Choice Question Answering
Rowan University.
Code-Guided Reasoning for Small Language Models: Evaluating Executable MCQA Scaffolds
May 12, 2026Prateek Biswas, Dhaval Patel, Vedant Khandelwal +2Multiple-Choice Question Answering Small Language Models
1IBM · 2Artificial Intelligence Institute at University of South Carolina
ReasonLab: A Controlled and Auditable Evaluation of Prompting Techniques for Multiple-Choice QA
May 7, 2026Inder Preet, Shuxin Lin, Dhaval PatelLLM Evaluation Language Model Generation Evaluation
IBM, Dublin · IBM, New York
Evaluating Large Language Models on Computer Science University Exams in Data Structures
Apr 25, 2026Edan Gabay, Yael Maoz, Jonathan Stahl +7LLM Evaluation Computer Science Education
Blavatnik School of Computer Science and AI
More Thinking, More Bias: Length-Driven Position Bias in Reasoning Models
Apr 21, 2026Xiao WangMultiple-Choice Question Answering CoT Reasoning
Pushing the Boundaries of Multiple Choice Evaluation to One Hundred Options
Apr 16, 2026Nahyun Lee, Guijin SonLLM Evaluation Multiple-Choice Question Answering
Chung-Ang University · Seoul National University
Hypothesis-Conditioned Query Rewriting for Decision-Useful Retrieval
Mar 19, 2026Hangeol Chang, Changsun Lee, Seungjoon Rho +2Retrieval-Augmented Generation Query Reformulation
Graduate School of AI, KAIST, Republic of Korea · School of Electrical Engineering, KAIST, Republic of Korea · Department of Industrial Engineering, Yonsei University, Republic of Korea
Inclusion-of-Thoughts: Mitigating Preference Instability via Purifying the Decision Space
Mar 15, 2026Mohammad Reza Ghasemi Madani, Soyeon Caren Han, Shuo Yang +1Multiple-Choice Question Answering Reasoning Consistency in Language Models
School of Computing and Information Systems, The University of Melbourne
Rethinking Multiple-Choice Questions for RLVR: Unlocking Potential via Distractor Design
Mar 13, 2026Xu Guo, Qiming Ge, Jian Tong +8RL for Language Model Reasoning Multiple-Choice Question Answering
Shanghai AI Laboratory · Shanghai Innovation Institute · Fudan University
Corpus Prevalence of Multiple-Choice Question Options
Feb 19, 2026Leonidas Zotos, Hedderik van Rijn, Malvina NissimMultiple-Choice Question Answering Generative AI in Education
Center for Language and Cognition, University of Groningen, Oude Kijk in ’t Jatstraat 26, Groningen, 9712 EK, The Netherlands · Department of Experimental Psychology, University of Groningen, Grote Kruisstraat 2/1, Groningen, 9712 TS, The Netherlands
Robustness assessment of large audio language models in multiple-choice evaluation
Oct 6, 2025Fernando López, Santosh Kesiraju, Jordi LuqueAudio-Language Model Evaluation Audio QA
Scientific Research, Telef´onica Innovaci´on Digital, Spain · Brno University of Technology, Czech Republic
Listening to the Wise Few: Query-Key Alignment Unlocks Latent Correct Answers in Large Language Models
Oct 3, 2024Eduard Tulchinskii, Kristian Kuznetsov, Laida Kushnareva +5Self-Attention Multiple-Choice Question Answering
Skolkovo Institute of Science and Technology · AI Foundation and Algorithm Lab · Moscow Institute of Physics and Technology +2

May 25, 2026J/K move · Enter open · S save

Second Guess: Detecting Uncertainty Through Abstention and Answer Stability in Small Language Models

Ashwath Vaithinathan Aravindan, Mayank Kejriwal

University of Southern California Los Angeles, 90007, California, United States of America · Information Sciences Institute 4676 Admiralty Way #1001, Los Angeles, 90292, California, United States of America

Confidence Estimation in Language Models Multiple-Choice Question Answering Small Language Models LLM Uncertainty Estimation LLM Abstention

Open paper AskPDF arXiv

Large language models often generate confident but incorrect answers rather than abstaining when uncertain. This problem is particularly acute for small language models (SLMs), where computational constraints and autonomous operation amplify the need for reliable uncertainty detection. We propose Second Guess, a lightweight, parameter-free prompting technique for abstention in multiple-choice question answering (MCQA) that is well-suited for SLMs. Our key empirical insight is that models which truly know an answer will select it consistently, while uncertain models exhibit unstable behavior when an ``I don't know'' option is added. Evaluated on four open models (2B-8B parameters) and four benchmarks, Second Guess achieves the highest composite risk improvement of 10.81%. Notably, it maintains an 8% composite risk improvement on fine-tuned models where entropy-based methods degrade, and improves most for lower-performing models. All code and results required to reproduce this work is available in https://github.com/Mystic-Slice/second-guess