Mmlu-Pro

Follow this topic Ask about this topic

Momentum

5 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 25

All topics

Measuring Collapse and Correction in Homogeneous-Panel LLM Debate
Sep 28, 2026Xin Li, Mengbing Liu, Chau YuenDebate Large Language Model Decisions
Nanyang Technological University
Disaggregated Quantization: Specializing LLM Prefill and Decode
Sep 22, 2026Andrei Panferov, Maximilian Kleinegger, Sweta Priyadarshi +2Large Language Model Quantization Quantizer
NVIDIA · ISTA
Temperature Fragility and the Conditional Benefits of Truncation Sampling
Sep 14, 2026Francesco La RosaTruncation Mmlu-Pro
University of Edinburgh
Turkish MMLU Pro: Traceable Option Augmentation and Its Validity Limits in Turkish Multiple-Choice Evaluation
Sep 14, 2026M. Ali BayramMultiple-Choice Questions Mmlu-Pro
Department of Computer Engineering, Yıldız Technical University İstanbul, Turkey
What Does MMLU Actually Measure? A Psychometric Audit of Difficulty Structure in Aggregate Benchmark Scores
Sep 8, 2026Dana Paquin, Riddhiman JainMmlu-Pro Psychometric Properties
Stanford University
Router Sensitivity Under Lightweight Fine-Tuning Identifies Prunable Experts in Mixture-of-Experts Models
Aug 8, 2026Ali Janati, Kaoutar El Maghraoui, Chengke Zou +2Experts Low-Rank Adaptation Framework
Data Science Institute, Columbia University, New York, NY, USA · Department of Computer Science, Columbia University, New York, NY, USA
Shorter Reasoning, Earlier Answers? An Evaluation of Reasoning Interfaces
Aug 4, 2026Francesca Carlon, Vincent Ginis, Andres AlgabaReasoning Traces Mmlu-Pro
Data Analytics Lab, Vrije Universiteit Brussel, Pleinlaan 5, 1050 Brussels, Belgium · imec-SMIT, Vrije Universiteit Brussel, Pleinlaan 9, 1050 Brussels, Belgium · School of Engineering and Applied Sciences, Harvard University, Cambridge, Massachusetts 02138, USA
Conformity Mitigations in Large Language Models Lie on a Single Resistance-Receptivity Frontier
Aug 3, 2026Zafar Hussain, Kristoffer NielboMmlu-Pro Conformity
Aarhus University
Understanding Tone-Dependent Inference Cost in Large Language Models
Jul 27, 2026Akhil Kumar, Om DobariyaInference Cost Mmlu-Pro
Pennsylvania State University University Park, PA, USA
Building a European Multilingual Evaluation Dataset: The MMLU Localisation Project within the EMT Network
Jul 20, 2026Pilar Sánchez-Gijón, Susana Valdez, Sofía Calvo Del Barrio +3Multilingual Mmlu-Pro
Universitat Autònoma de Barcelona / Spain · Leiden University/ the Netherlands · Directorate-General for Translation of the European Commission / Belgium
evalci: A Python Library for Statistically Rigorous Comparison of Language Model Evaluations
Jul 5, 2026Shreyas K ChandrahasLarge Language Model Evaluation Mmlu-Pro
Capability Provenance in Language Models: A Case Study in Social Reasoning
Jun 17, 2026Glenn Matlin, Chandreyi Chakraborty, Saehee Eom +8Social Reasoning Reasoning Skills
Georgia Institute of Technology, College of Computing · MATS Program
Decompose Sparsely Where You Should, Absorb Densely Where You Should No
Jun 12, 2026Ruixuan Deng, Zehao Jin, Zekun Wang +1Improving Sparse Autoencoders Sparsity
School of Psychology Georgia Institute of Technology · College of Computing Georgia Institute of Technology · School of Interactive Computing Georgia Institute of Technology
Resolution Diagnostics for Paired LLM Evaluation
May 28, 2026Anany KotawalaLarge Language Model Evaluation Spearman Correlation
Princeton University, Princeton, NJ, USA.
Law of Neural Interaction: Depth-Width Shape, Interaction Efficiency, and Generalization
May 27, 2026Wenjie Sun, Jinning Yang, Shuai Zhang +1Interaction Prediction Mmlu-Pro
The Chinese University of Hong Kong, Shenzhen · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences · New Jersey Institute of Technology
K-Quantization and its Impact on Output Performance
May 19, 2026Robin Baki Davidsson, Pierre NuguesLarge Language Model Quantization Mmlu-Pro
Lund University · Lund, Sweden
Bias and Uncertainty in LLM-as-a-Judge Estimation
May 7, 2026James FiedlerLlm-As-A-Judge Raw Judge Outputs
Indeed Inc.
Decodable but Not Corrected by Fixed Residual-Stream Linear Steering: Evidence from Medical LLM Failure Regimes
May 7, 2026Ming LiuLarge Language Models Fail Decoding
Amazon
Benchmarking Local Language Models for Social Robots using Edge Devices
May 4, 2026Dorian Lamouille, Matevž B. Zorec, Farnaz Baksh +1Small Large Language Models Large Language Model Benchmarks
University of Tartu Tartu, Estonia · Institute of Computer Science University of Tartu Tartu, Estonia · Institute of Technology University of Tartu Tartu, Estonia
Option-Order Randomisation Reveals a Distributional Position Attractor in Prompted Sandbagging
Apr 29, 2026Jon-Paul CacioliMmlu-Pro Position Bias
Independent Researcher, Melbourne, Australia
Multilingual Refusal Alignment for Safer Large Language Models
Apr 24, 2026Aleksandra Krasnodębska, Wojciech Kusa, Aldo LipaniMultilingual Safety Large Language Model Alignment
NASK National Research Institute, Warsaw, Poland · University College London, London, UK
Domain-level metacognitive monitoring in frontier LLMs: A 33-model atlas
Apr 21, 2026Jon-Paul CacioliMmlu-Pro Multi-Turn Benchmark
Independent Researcher · Melbourne, Australia
Does Self-Consistency Improve the Recall of Encyclopedic Knowledge?
Apr 21, 2026Sho Hoshino, Ukyo Honda, Peinan ZhangSelf-Consistency Factual Recall
CyberAgent
Modeling LLM Unlearning as an Asymmetric Two-Task Learning Problem
Apr 16, 2026Zeguan Xiao, Siqing Li, Yong Wang +4Large Language Model Unlearning Large Language Model Training
1Shanghai University of Finance and Economics · 2Southern University of Science and Technology · 3Alibaba Group +2
UrduMMLU: A Massive Multitask Benchmark for Urdu Language Understanding
Date pendingAhmer Tabassum, Sarfraz Ahmad, Hasan Iqbal +3Urdu Multilingual Benchmark
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)

Sep 28, 2026, cs.CLJ/K move · Enter open · S save

Measuring Collapse and Correction in Homogeneous-Panel LLM Debate

Xin Li, Mengbing Liu, Chau Yuen

Nanyang Technological University

Debate Large Language Model Decisions Multi-Agent Large Language Model Systems Mmlu-Pro Collapse Multiple-Choice Questions+1

Open paper AskPDF arXiv

Multi-agent large language model (LLM) debate is often evaluated by whether final answers improve, but movement is not necessarily improvement: the same discussion can rescue an initially wrong majority or destroy an initially correct one. Standard final-accuracy evaluations conflate these opposing mechanisms. We introduce an auditable protocol for homogeneous debate on multiple-choice questions (MCQs) that records each run as a transition ledger over collapse, correction, onset, and signed intervention utility. On 6,925 MMLU-Pro debates, the protocol identifies 253 collapses and a parallel correction ledger that changes how interventions should be judged. Replay experiments reveal the central tradeoff: a leave-one-model-out probe-gated freeze prevents 29 collapses but loses 108 corrections under equal weights, so collapse prevention alone can recommend the wrong policy. A compact pre-debate 8-probe screen is a triage signal: its unadjusted family-level association with conditional-collapse risk is high (G=7, Spearman rho=0.893, exact two-sided p=0.0123), but initial-majority accuracy is a close comparator (rho=0.821; family partial rho=0.767, p=0.0877), so we do not treat it as calibrated or capability-adjusted prediction. Round-level traces localize many collapses to the first debate round, where early disagreement can precede both harmful cascades and useful recovery. We release replayable schemas, coders, audits, cost cards, and zero-API rebuild scripts so future model-scaffold rows can be compared under the same denominators and signed utility ledger.

Momentum

Related topics

Latest papers 25