Language Model Ensembles

Follow this topic Ask about this topic

Momentum

15 papers in the last four weeks, up 400% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 93

All topics

JudgeMoE: Distributional Aggregation for LLM-as-a-Judge
Oct 5, 2026Yiqi Liu, Joseph James, Yang Wang +3LLM Evaluation LLM-as-a-Judge
The University of Manchester · The University of Sheffield · University of Pittsburgh +1
From Discovery to Decision: Finite-Budget Recoverability in LLM Voting
Oct 1, 2026Shaoang Li, Jian LiLLM Inference Efficiency LLM Inference
Stony Brook University
Adapter Thickets: Splitting an RLVR Budget Beats Concentrating It
Oct 1, 2026Jonathan Williams, Esin Tureci Karthik R. NarasimhanRL for Language Model Reasoning Low-Rank Adaptation
Princeton University
Beyond Leaderboards: Tokenomics of Agentic Small Language Model Ensembles
Oct 1, 2026Alexei N. Skurikhin, Emily M. Taylor, Nathan A. DeBardelebenLLM Inference Efficiency Language Model Generation Evaluation
Los Alamos National Laboratory Los Alamos, New Mexico, USA
RAIM: Robust Aggregation of Inexpensive Models for Hallucination Detection
Sep 30, 2026Elia Onofri, Roberto Di PietroLLM Evaluation LLM-as-a-Judge
Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division, King Abdullah University of Science and Technology (KAUST) Thuwal 23955, Saudi Arabia
Reach Into The CHOIR: Free-List Elicitation Uncovers Distinct Model Voices in LLM Ensembles
Sep 29, 2026Ben Wigler, Maria TsfasmanLanguage Model Ensembles Language Model Probing
LoveMind AI, New York, USA
Which Models Work Well Together? Measuring Heterogeneity for LLM Team Selection
Sep 29, 2026Liangyu Teng, Hengsong Liu, Juncen Guo +4Language Model Ensembles Ensemble Diversity
Fudan Institute on Networking Systems of AI
MERGE: Multi-LLM Ensemble for Retrieval via Generative Enrichment
Sep 29, 2026Tzu-I Ho, Yung-Yu Shih, Shang-Yu Su +2LLM Prompting Automatic Prompt Optimization
University of Waterloo · National Taiwan University · Rakuten Group, Inc. +1
Modular Norm RandOpt: Population-Efficient Ensembling through Architecture-Aware Perturbations
Sep 22, 2026Kirato Yoshihara, Hiroaki HamadeLanguage Model Ensembles
The University of Osaka
How Many Humans Are 32 LLM Judges Worth?
Sep 18, 2026Chao Li, Yingying Yu, Yunfeng LiAnnotator Disagreement LLM-as-a-Judge
Tsinghua University · University College London · Hebei University of Economics and Business
Mo' Models, Mo' Problems: How to best select model pools when designing Multi-Agent Systems
Sep 15, 2026Sara Vera Marjanović, Jiacheng Xu, Aleksandr Laptev +3Model Selection Multi-Agent LLM Systems
University of Copenhagen · Work done during internship at NVIDIA · NVIDIA
The Universe of Universes: Benefit Yield Functions, Implosion Thresholds, and Infrastructure-Aware Optimization in Multi-LLM Systems
Sep 14, 2026Danielle Franklin, Vasu Raj JainLanguage Model Ensembles Multi-LLM Collaboration
Humanity + AI, Inc.
Can We Trust LLM Judges: A Study of Capability-Dependent Biases and Multi-Judge Ensemble for Bias Calibration
Sep 14, 2026Gemma Zhang, Prachi Badarayani, Asmi Kumar +2LLM Evaluation LLM-as-a-Judge
Microsoft · Massachusetts Institute of Technology
LLM-Enhanced Dual-Branch Learning for Large-Scale Multi-Label Text Classification
Sep 14, 2026Hui Ye, Jing Zhang, Xiulong Yang +1Multi-Label Classification Language Model Ensembles
Department of Computer Science, Georgia State University, Atlanta, GA 30303, USA · Amazon, San Diego, CA, USA · School of Computer Science, Central China Normal University, Wuhan 430079, China
Bag of Tricks or Bag of Myths? Reducing Modeling Complexity with Task Knowledge in Explainable Suicide Risk Assessment
Sep 7, 2026Shlok Shelat, Shrey Salvi, Souvik Roy +2Language Model Ensembles Suicide Risk Modeling
Indian AI Research Organisation · Ahmedabad University · University of Maryland, Baltimore County
Retrieval-Augmented Multi-Prompt Ensemble for Minor-Grain Breeding Information Extraction
Sep 7, 2026Hang Zhao, Jiahao WangRetrieval-Augmented Generation Document Information Extraction
Zhengzhou University · Harbin Institute of Technology (Shenzhen)
Inferred Generative-Process Diversity Predicts Correlated Failure Across Language Models
Sep 3, 2026Ross Tieman, Evan MarkouLLM Reliability Language Model Ensembles
Fenner School of Environment & Society · Australian National University · School of Computing
Unifying Conformal Language Tasks with In-Context Ensembles
Sep 2, 2026Xiao Shi Huang, Chen-Yuan Lin, Bruce Kuwahara +2Conformal Prediction Language Model Ensembles
Signal 1 AI · Layer 6 AI
When Do Supervised UQ Ensembles Improve LLM Hallucination Detection? A Robustness Study
Aug 25, 2026Mohit Singh Chauhan, Vipin Gyanchandani, Dylan BouchardLLM Hallucination Detection LLM Uncertainty Estimation
CVS Health®, Wellesley, MA, USA
Opportunity Is Not Realizability: Selection-Valid Diagnostics for Multi-LLM Routing
Aug 8, 2026Ibne Farabi Shihab, Abu Sa-Adat Mohamed Moon-Im Al Ahsan, Md Najmus SwaqeebModel Selection LLM Evaluation
Department of Computer Science, Iowa State University · Department of Computer Science & Engineering, BRAC University
Agreement Before Diversity: Verification-First Complementarity for Heterogeneous Language-Model Coordination
Aug 5, 2026Ruitong Li, Binjie Guo, Aisheng Mo +3LLM Evaluation LLM Answer Verification
1The University of Hong Kong · Zhejiang University · 3Independent Researcher
HomoEnsNER: Does Language Alignment Outperform Architectural Complexity in Gujarati Named Entity Recognition?
Aug 4, 2026Chandrakant K. BhogayataNamed Entity Recognition Language Model Ensembles
Maharaja Krishnakumarsinhji Bhavnagar University, Bhavnagar (Gujarat, India)
Floor, Ceiling, and the Fusion Gap: How Much of Crowd Reading Attention Can Machines Predict?
Aug 3, 2026Kazuki Nakayashiki, Keisuke WatanabeLLM Evaluation Language Model Ensembles
Glasp Inc.
Sixteen models, fewer than two voices: measuring ensemble dispersion where no answer is uniquely correct
Jul 31, 2026Mario Vega-Barbas, Lidia Mora-Valenciano, Iván Pau +2Language Model Ensembles Clinical Language Model Evaluation
ETSIS de Telecomunicación, Universidad Politécnica de Madrid, Madrid, Madrid, Spain · Independent Researcher, Guadalajara, Spain · Department of Clinical Science, Intervention and Technology, Karolinska Institutet, Huddinge, Stockholm, Sweden +4
Reasoning Consensus: Structural Ensembling of LLM Reasoning via Weighted DAG Aggregation
Jul 30, 2026Amruta Parulekar, Jinu Lee, Dilek Hakkani-Tür +1LLM Reasoning with Graphs LLM Interpretability
University of Illinois Urbana-Champaign
RareLens: Towards End-to-End Rare Disease Care via Aligning Divergent Large Language Model Reasoning
Jul 25, 2026Xi Chen, Hongru Zhou, Shiyu Feng +24Rare Disease Diagnosis Healthcare
Sports Medicine Center, Department of Orthopedics and Orthopedic Research Institute, West China Hospital, Sichuan University, Chengdu, China · Center for Cleft Lip and Palate Treatment, Plastic Surgery Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China · School of Data Science, The Chinese University of Hong Kong, Shenzhen (CUHKSZ), Shenzhen, China +13
Verbalized Particle Posterior: Bayesian Inference over Natural Language Hypotheses
Jul 25, 2026Yan Zhang, Shikan Lian, Shibo LiBayesian Inference Language Model Ensembles
Department of Computer Science, Florida State University
Are Diversity Metrics Measuring Diversity? A Capability-Controlled Audit of Majority-Vote Gain in LLM Ensembles
Jul 22, 2026Donghwan KimLLM Evaluation Language Model Ensembles
Aidentyx Inc., San Jose, CA, USA
PoTRE: Test-Time Reasoning inspired by Cognitive Heterogeneity
Jul 22, 2026Anmol Kankariya, Sercan Ö. ArıkTest-Time Scaling Language Model Ensembles
Applied AI, Google Cloud
Small, Free, and Effective: Orchestrating Open-Weight Small Language Models to Outperform Single LLM for Malware Analysis
Jul 22, 2026Adel ElZemity, Shujun Li, Budi AriefMulti-Agent LLM Systems Small Language Models
University of Kent, Canterbury, United Kingdom

Oct 5, 2026, cs.CLJ/K move · Enter open · S save

JudgeMoE: Distributional Aggregation for LLM-as-a-Judge

Yiqi Liu, Joseph James, Yang Wang, Kun Zhao+2

The University of Manchester · The University of Sheffield · University of Pittsburgh+1

LLM Evaluation LLM-as-a-Judge Ensemble Learning Language Model Ensembles

Open paper AskPDF arXiv

When an LLM judge scores an output, its score distribution retains uncertainty and disagreement information that is lost after scalar compression. We introduce JudgeMoE, a lightweight aggregator that assigns example-specific weights to cached judge score distributions and fuses them before computing a final score. A protocol study shows that score-range choice is unstable across judge--dataset settings and that soft scoring usually outperforms hard decoding. On the original 10-cell benchmark, JudgeMoE improves mean Spearman over uniform log pooling by

+0.079

. Applying the same configuration to six additional cells yields a

+0.0393

mean gain over the strongest local single judge across 16 cells, with positive differences in 12/16 cells and a one-sided Wilcoxon signed-rank

p=0.0091

. Validation-based analyses further show that the preferred aggregation method depends on the task and judge pool.

Momentum

Related topics

Latest papers 93