Competition

Follow this topic Ask about this topic

Momentum

7 papers in the last four weeks, down 12% on the four weeks before. 0.1% of all new papers.

Jul 6Week of Sep 21

Latest papers 70

All topics

On Benchmark Hacking in ML Contests: Modeling, Insights and Design
Apr 24, 2026Xiaoyun Qiu, Yang Yu, Haifeng XuMle-Bench Lite Game Theory
Department of Economics, Dartmouth College. · Sloan & CSAIL, Massachusetts Institute of Technology. · Department of Computer Science, University of Chicago.
MathDuels: Evaluating LLMs as Problem Posers and Solvers
Apr 23, 2026Zhiqiu Xu, Shibo Jin, Shreya Arya +1Competition Solvers
Department of Computer and Information Science, University of Pennsylvania · Rabdos AI · Department of Mathematics, University of Pennsylvania
Performance Anomaly Detection in Athletics: A Benchmarking System with Visual Analytics
Apr 23, 2026Blessed Madukoma, Prasenjit MitraAthleticism Competition
Carnegie Mellon University Africa · Kigali, Rwanda
Adversarial Arena: Crowdsourcing Data Generation through Interactive Competition
Apr 20, 2026Prasoon Goyal, Sattvik Sahai, Michael Johnston +14Large Language Model Generation Adversarial Robustness
Amazon Nova Responsible AI
From Competition to Coopetition: Coopetitive Training-Free Image Editing Based on Text Guidance
Apr 17, 2026Jinhao Shen, Haoqian Du, Xulu Zhang +2Image Editing Competition
Department of Computing, Hong Kong Polytechnic University, Hong Kong, China · Department of Computer Science, Sichuan University, Chengdu 610017, China
Reasoning and Tool-use Compete in Agentic RL:From Quantifying Interference to Disentangled Tuning
Feb 1, 2026Yu Li, Mingyang Yi, Xiuyu Li +6Agentic Reinforcement Learning Agentic Reasoning
School of Information, Renmin University of China, Beijing, China · Bytedance Inc., Beijing, China · Bytedance Inc., San Jose, USA
The Challenger: When Do New Data Sources Justify Switching Machine Learning Models?
Dec 20, 2025Vassilis Digalakis, Christophe Pérignon, Sébastien Saurin +1Credit Risk Prediction Data-Driven
Boston University · HEC Paris
When and Why Does Multi-Agent Debate Fail and Does It Really Underperform?
Oct 23, 2025Yongqiang Chen, Gang Niu, James Cheng +2Multi-Agent Debate Competition
1The Chinese University of Hong Kong · 2RIKEN Center for Advanced Intelligence Project · 3Hong Kong Baptist University +1
LM Fight Arena: Benchmarking Large Multimodal Models via Game Competition
Oct 10, 2025Yushuo Zheng, Tongrui Ye, Zicheng Zhang +3Chatbot Arena Large Multimodal Models
Shanghai Jiao Tong University, Shanghai, China · Shanghai AI Lab, Shanghai, China · Tongji University, Shanghai, China
ContestTrade: A Multi-Agent Trading System Based on Internal Contest Mechanism
Aug 1, 2025Rui Sun, Li Zhao, Zuoyou Jiang +5Competition Deep Research
1Stepfun · 2FinStep

Apr 24, 2026, econ.GNJ/K move · Enter open · S save

On Benchmark Hacking in ML Contests: Modeling, Insights and Design

Xiaoyun Qiu, Yang Yu, Haifeng Xu

Department of Economics, Dartmouth College. · Sloan & CSAIL, Massachusetts Institute of Technology. · Department of Computer Science, University of Chicago.

Mle-Bench Lite Game Theory Competition Nash Equilibrium Insights

Open paper AskPDF arXiv

Benchmark hacking refers to tuning a machine learning model to score highly on certain evaluation criteria without improving true generalization or faithfully solving the intended problem. We study this phenomenon in a generic machine learning contest, where each contestant chooses two types of effort: creative effort that improves model capability as desired by the contest host, and mechanistic effort that only improves the model's fitness to the particular task in contest without contributing to true generalization. We establish the existence of a symmetric monotone pure strategy equilibrium in this competition game. It also provides a natural definition of benchmark hacking in this strategic context by comparing a player's equilibrium effort allocation to that of a single-agent baseline scenario. Under our definition, contestants with types below certain threshold (low types) always engage in benchmark hacking, whereas those above the threshold do not. Furthermore, we show that more skewed reward structures (favoring top-ranked contestants) can elicit more desirable contest outcomes. We also provide empirical evidence to support our theoretical predictions.

Momentum

Related topics

Latest papers 70