Model Evaluation

Follow this topic Ask about this topic

Recent momentum

-70%

3 papers in the last 28 days · 0.1% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-14

2 new papers

A weekly snapshot of new work published in Model Evaluation.

Period ending 2026-09-07

3 new papers

A weekly snapshot of new work published in Model Evaluation.

62 papers

Latest in Model Evaluation

Open your feed →

The widening evaluation gap in medical large language model research 2023 to 2026
Sep 11, 2026Raad Bin Tareaf, Murad Al-Rajab, Samia LoucifModel Evaluation Empirical Study
Rethinking Sign Language Translation: The Impact of Signer Dependence on Model Evaluation
Sep 7, 2026Keren Artiaga, Sabyasachi Kamila, Haithem Afli +2Sign Language Translation Online Signature Verification
Foundation Models Meet Agriculture: Challenges Beyond Pretraining
Aug 31, 2026Vishal Nedungadi, Xingguo Xiong, Marc Rußwurm +1Geospatial Foundation Models Usda Cropland Data Layer
Incremental Evaluation and Training in Relational Deep Learning
Aug 13, 2026Jakub Peleška, Gustav ŠírEfficient Continual Learning Relational
Robust Ambiguity Detection (RAD) From Model- and Feature-Space Consistency
Aug 12, 2026Manya Singh, Mark T. Keane, Arjun PakrashiAmbiguity Model Evaluation
Evaluating Machine Learning Models for Post-Wildfire Debris-Flow Prediction
Aug 5, 2026Quinn Ledingham, Zhengsen Xu, Yimin Zhu +6Wildfire Prediction Accurate Flood Prediction
Dynamically Allocating Evaluation Effort for Model Ranking
Aug 4, 2026Vilém Zouhar, Julia Kreutzer, Alon Lavie +4Model Evaluation Evaluation Benchmarks
NOMADD: Numerical Optimization of Models Adapting to Data Drift
Aug 3, 2026Swapn Shah, Keith BurghardtState Drift Tabular Prior-Data Fitted Network
Object Detection for Autonomous Driving in Chinese Rural Scenes: An Experimental Study on Real-Synthetic Data Mixing and Model Evaluation
Jul 29, 2026Danning Zhu, Ziyan Lin, Jing WuYolo26 Object Detection
BayesAME: Bayesian Active Model Evaluation
Jul 29, 2026Paula Cordero Encinar, Taylan Cemgil, Arnaud Doucet +2Geometry-Aware Uncertainty Coresets Model Selection
Enhancing Automated Machine Learning via Homogeneous Train-Test Splitting Methods
Jul 29, 2026Yearn Tan Yin Tze, Charles GrelloisModel Evaluation Split Learning
Certified in Theory, Broken in Practice: Assumption Gaps in Cryptographic Model Certification
Jul 23, 2026Carter Luck, Olive Franzese-McLaughlin, Elisaweta Masserova +3Algorithm Auditing Model Evaluation
Evaluating Federated Pre-Training: On the Reliability of Downstream Fine-Tuning and Intrinsic Evaluation
Jul 20, 2026Claudia Grosser, Maike Heuer, Denis Krompass +1Federated Fine-Tuning Downstream Fine-Tuning
Can We Trust Item Response Theory for AI Evaluation?
Jul 16, 2026Han Jiang, Sunbeom Kwon, Jinwen Luo +2Item Response Theory Artificial Intelligence Benchmarks
A Novel Method to Evaluate Models on Unreliable, Noisy and Inconsistent Labels: Adaptive Resolution Label Aggregation (ARLA)
Jul 13, 2026Natasha Randall, Gernot HeisenbergNoisy Labels Model Evaluation
Stop Guessing When to Stop Testing: Efficient Model Evaluation with Just Enough Data
Jul 9, 2026Ofir Arviv, Kristjan Greenewald, Yotam Perlitz +3Model Evaluation Benchmark Suite
Full-range Binary Classifier Calibration for Stable Model Updates in Production
Jul 6, 2026Konstantin BerlinFalse Positive Rate Recalibration
CollabEval: Statistically Efficient Collaborative Model Evaluation via Matrix Completion
Jul 6, 2026Adam Fisch, Daniel Deutsch, Joshua Maynez +5Model Evaluation Tensor Completion
Multistage Defer Trees for Hybrid Interpretability: If at First You Can't Succeed, Tree Again
Jun 30, 2026Zakk Heile, Hayden McTavish, Margo Seltzer +1Decision Trees Interpretable Machine Learning
HERO: Improving the Reliability and Sensitivity of Generative Model Evaluation Using Historical Data
Jun 29, 2026Xinrui Ruan, Zhenyu Zhao, Waverly Wei +4Model Evaluation Human Annotations
Towards Evaluating Data Priors for Tabular Foundation Models
Jun 28, 2026Zeynep Türkmen, Kürşat Kaya, Alexander Pfefferle +1Tabular Foundation Models Priors
Model selection with proper scoring rules on data sets of time series: prefer the mean scaled score
Jun 23, 2026Giorgio Corani, Stefano Damato, Dario Azzimonti +1Model Selection Retrieval-Augmented Forecasting
You Don't Need to Run Every Eval
Jun 22, 2026Yuchen Zeng, Dimitris PapailiopoulosModel Evaluation Scorer
MassSpecGym in the Wild: Uncovering and Correcting Evaluation Pitfalls in AI-Driven Molecule Discovery
Jun 17, 2026Hongxuan Liu, Roman Bushuiev, Ivy Lightheart +12Mle-Bench Lite Model Evaluation
Beyond the Training Distribution: Evaluating Predictions Under Distribution Shift and Selection Bias
Jun 12, 2026Annie Ulichney, Amanda CostonCovariate Shift Selection Bias
AfriSUD: A Dependency Treebank Collection for Evaluating Models on African Languages
Jun 10, 2026Happy Buzaaba, Cheikh Mouhamadou Bamba Dione, David Ifeoluwa Adelani +15African Languages Prague Dependency Treebank
Rank Intervals for Leaderboards: A Hierarchical Framework for Model Evaluation
Jun 7, 2026Bitya Neuhof, Yuval BenjaminiGeneralizable Ranking Purpose Leaderboard
RECAP: Regression Evaluation for Continual Adaptation of Prompts
Jun 4, 2026Harsh Deshpande, Kushal Chawla, Sangwoo Cho +2Prompt Model Evaluation
Bridging Domain Expertise and Generalization for Performance Estimation
Jun 4, 2026Shuxuan Li, Zhilin Zhao, Quyu Kong +1Distribution Shifts Parameter-Efficient Adaptation
New Benchmarking Shows Limited Generalization Power of TCR Antigenic Epitope Prediction Models
Jun 3, 2026Yiming Liao, Yiheng Li, Ning Jiang +2Epitope Prediction Antibody
Measuring Model Robustness via Fisher Information: Spectral Bounds, Theoretical Guarantees, and Practical Algorithms
Jun 3, 2026Chong Zhang, Xiang Li, Jia Wang +2Fisher Information Model Evaluation
When Offline Selectors Cannot Beat the Best Single Model: A Diagnostic Study on edX Dropout Prediction
Jun 2, 2026Tyler Crosse, Alan Nadelsticher Ruvalcaba, Dustin Khang LeDuc +3Predictors Dropout
Predicting Inference-Time Scaling Gains from Labeled Validation-Set Output Statistics
Jun 2, 2026Luyang Zhang, Jingyan LiBest-Of-N Inference-Time
TeachObs: A Human-Validated Benchmark for Multimodal Teaching Observation and Model Evaluation
May 29, 2026Yeil Jeong, Youngjin Yoo, Jiyoung Bae +5Pedagogical Frameworks Human Annotations
A Multi-dimensional Framework for Evaluating Generalization in EEG Foundation Models
May 27, 2026Aditya Kommineni, Emily Zhou, Kleanthis Avramidis +2Electroencephalography Foundation Models Wireless Foundation Models
High Performance, Low Reliability: Uncertainty Benchmarking for Tabular Foundation Models
May 27, 2026José Lucas De Melo Costa, Fabrice Popineau, Arpad Rimmel +1Tabular Foundation Models Uncertainty Quantification
Explaining Tabular Foundation Model Differences Through Meta-Features
May 27, 2026Markus Herre, Andrej Tschalzev, Sascha Marton +1Tabular Foundation Models Model Evaluation
The Missing Piece in Pre-trained Model Evaluation: Reward-Guided Decoding Unlocks Task-Oriented Behavior Without Parameter Updates
May 27, 2026Shaobo Wang, Guo Chen, Ziyue Wang +5Large Language Model Post-Training Large Language Model Decoding
Evaluating Local Explainability Metrics for Machine Learning Models on Tabular Data
May 26, 2026Tomás Pereira, João Vitorino, Eva Maia +1Explainability Concept-Based Explainability
Selective Test-Time Compute Scaling for Click-Through Rate Prediction via Uncertainty-Triggered Feature Path Exploration
May 24, 2026Moyu Zhang, Yun Chen, Yujun Jin +3Click-Through Rate Prediction Test Time
Towards Evaluation Engineering: An Empirical Study of ML Evaluation Harnesses in the Wild
May 22, 2026Zhimin Zhao, Zehao Wang, Abdul Ali Bangash +2Model Evaluation Engineering
Learning to Evaluate: Cost-Effective Model Evaluation on Unlabeled Data with Meta-Learning
May 22, 2026Trinh Pham, Viet Huynh, Hongzhi Yin +2Model Evaluation Unlabeled Data
Ensembling Tabular Foundation Models - A Diversity Ceiling And A Calibration Trap
May 18, 2026Aditya Tanna, Yash Desai, Pratinav Seth +3Tabular Foundation Models Ensemble
Foundation Models for Credit Risk Prediction: A Game Changer?
May 18, 2026Bart Baesens, Andreas Goethals, Stefan Lessmann +10Credit Risk Prediction Tabular Foundation Models
Training ML Models with Predictable Failures
May 14, 2026Will Schwarzer, Scott NiekumEarly Failure Prediction Model Evaluation
NodeSynth: Socially Aligned Synthetic Data for AI Evaluation
May 14, 2026Qazi Mamunur Rashid, Xuan Yang, Zhengzhe Yang +5Synthetic Data Generative Artificial Intelligence
Does Your Wildfire Prediction Model Actually Work, or Just Score Well?
May 14, 2026Yangshuang Xu, Yuyang Dai, Liling Chang +2Wildfire Prediction Model Evaluation
Quantifying Sensitivity for Tree Ensembles: A symbolic and compositional approach
May 13, 2026Ajinkya Naik, Chaitanya Garg, S. Akshay +2Decision Trees Ensemble
What properties of reasoning supervision are associated with improved downstream model quality?
May 13, 2026Mikołaj Langner, Dzmitry Pihulski, Jan Eliasz +5Table Reasoning Datasets Model Evaluation
A Transfer Learning Evaluation of Deep Neural Networks for Image Classification
May 12, 2026Nermeen Abou Baker, Nico Zengeler, Uwe HandmannTransfer Learning Imagenet
FLAM: Evaluating Model Performance with Aggregatable Measures in Federated Learning
May 8, 2026Fabian Stricker, Jose A. Peregrina, David Bermbach +1Model Evaluation Fairness-Aware Learning
Query-efficient model evaluation using cached responses
May 8, 2026Hayden Helm, Ben Johnson, Carey PriebeModel Evaluation Predictive Performance
Correct Code, Vulnerable Dependencies: A Large Scale Measurement Study of LLM-Specified Library Versions
May 7, 2026Chengjie Wang, Jingzheng Wu, Xiang Ling +2Vulnerability Detection & Repair Code Quality
PACE: Prune-And-Compress Ensemble Models
May 7, 2026Fabian Akkerman, Julien Ferry, Théo Guyard +1Ensemble Model Evaluation
Deployment-Relevant Alignment Cannot Be Inferred from Model-Level Evaluation Alone
May 6, 2026Varad Vishwarupe, Nigel Shadbolt, Marina Jirotka +1Model Evaluation Artificial Intelligence Alignment
PLMGH: What Matters in PLM-GNN Hybrids for Code Classification and Vulnerability Detection
Apr 28, 2026Mohamed Taoufik Kaouthar El Idrissi, Edward Zulkoski, Mohammad HamdaqaCode Quality Gnn-Based Detectors
SCURank: Ranking Multiple Candidate Summaries with Summary Content Units for Enhanced Summarization
Apr 21, 2026Bo-Jyun Wang, Ying-Jia Lin, Hung-Yu KaoSummarization Model Evaluation
FLARE: Task-agnostic embedding model evaluation through a normalization process
Apr 19, 2026Jingzhou Jiang, Yixuan Tang, Yi Yang +1Vector Embeddings Model Evaluation
Systematic Capability Benchmarking of Frontier Large Language Models for Offensive Cyber Tasks
Apr 18, 2026Tyler H. Merves, Michael H. Conaway, Joseph M. Escobar +2Cybersecurity Language Model Evasion Attacks
DEEP-GAP: Deep-learning Evaluation of Execution Parallelism in GPU Architectural Performance
Apr 16, 2026Kathiravan PalaniappanRtx Parallel Decoding and Training

Sep 11, 2026 · cs.CLJ/K move · Enter open · S save

The widening evaluation gap in medical large language model research 2023 to 2026

Raad Bin Tareaf, Murad Al-Rajab, Samia Loucif

Large language models are superseded every few quarters; clinical evidence takes years. We asked whether medical research is keeping pace with the systems it evaluates. PubMed returned 11,628 records for January 2023 to June 2026 across fourteen clinical domains, growing 45-fold; 2.5% used a randomised, controlled or prospective design. Evaluation lag, from a study's newest named model release to its own publication, widened from 1.33 to 6.08 quarters. Because discontinued models age mechanically, we benchmarked this against a counterfactual holding model composition fixed: migration to newer systems offset only 56% of the drift (95% CI 50-65). Randomised trials evaluated models a median 4.6 quarters older than other designs (P = 3 x 10^-19), yet among studies naming a model still under development no design differed from any other; 62% of randomised trials evaluated a discontinued family. Rigour and currency are in tension, and that tension reflects model selection rather than research timelines.

Model Evaluation Empirical Study

Open paper Ask about it PDF