Olympiad-Level Problem

Follow this topic Ask about this topic

Momentum

2 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 17

All topics

Learning Strategies to Break Judges
Sep 27, 2026Guruprerana Shabadi, Aaditya Naik, Rajeev Alur +1Judges Agentic Evaluations
University of Pennsylvania · Oracle
An Open Recipe for IMO Gold: Training Nemotron for Olympiad Mathematics
Sep 12, 2026Ivan Moshkov, Stephen Ge, George Armstrong +3Olympiad-Level Problem Proof
NVIDIA
StudyBench: Can Self-Evolution Squeeze Textbooks for Olympiad Capability?
Sep 1, 2026Yinghao Chen, Zixi Chen, Bingxiang He +7Self-Evolution Olympiad-Level Problem
Tsinghua University · Zhejiang University
ScienceArena: Benchmarking LLMs on Latest Scientific Olympiad Competitions
Aug 31, 2026Guangxiang Zhao, Qilong Shi, Xusen Xiao +13Olympiad-Level Problem Scientific Discovery
1Qiyuan Tech · 2Tsinghua University · 3The University of Hong Kong +1
Right Answer, Wrong Method: Shortcut Hacking Misleads the Evaluation of LLM Reasoning on Frontier Science Benchmarks
Aug 3, 2026Xuan Ren, Weiqi Zhai, Tianle Pu +4Reasoning Benchmark Large Language Model Benchmarks
1Alibaba Group · 2Alibaba DAMO Academy
AIMO Interpretability Challenge
Jul 15, 2026Michal Štefánik, Philipp Mondorf, Andreas Waldis +11Olympiad-Level Problem Interpretability
National Institute of Informatics, Japan · Munich Center for Machine Learning / MaiNLP LMU · University of Tübingen +6
Conformal calibration and look-elsewhere effect in anomaly detection for new-physics searches
Jun 11, 2026Jack Y. Araz, Michael SpannowskyAnomaly Score Olympiad-Level Problem
Department of Physics and Astronomy, University College London, London, WC1E 6B, UK · Department of Engineering, City St. George’s, University of London, London, EC1V 0HB, UK · Institute for Theoretical Physics, Campus Süd, Karlsruhe Institute of Technology (KIT), D-76128 Karlsruhe, Germany +1
WorldOlympiad: Can Your World Model Survive a Triathlon?
Jun 9, 2026Yuke Zhao, Wangbo Zhao, Weijie Wang +8Video World Models Generative Video Models
Zhejiang University · The Hong Kong University of Science and Technology · DAMO Academy, Alibaba Group +2
ComBench: A Benchmark for Rigorous Proof Reasoning and Constructive Realization in Olympiad-Level Combinatorics
Jun 9, 2026Shunkai Zhang, Haoran Zhang, Yun Luo +15Olympiad-Level Problem Combinations
2Peking University · 1Shanghai AI Laboratory · 3Shanghai Jiao Tong University +2
Using Aristotle API for AI-Assisted Theorem Proving in Lean 4: A Formalisation Case Study of the Grasshopper Problem
May 19, 2026Gabriel Rongyang LauTheorem Proving Olympiad-Level Problem
Nanyang Technological University, Singapore
Physics-R1: An Audited Olympiad Corpus and Recipe for Visual Physics Reasoning
May 13, 2026Shan YangOlympiad-Level Problem Multimodal Evaluation
Independent Researcher
Achieving Gold-Medal-Level Olympiad Reasoning via Simple and Unified Scaling
May 13, 2026Yafu Li, Runzhe Zhan, Haoran Zhang +25Olympiad-Level Problem Curriculum
Shanghai AI Laboratory · The Chinese University of Hong Kong · Shanghai Jiao Tong University +2
Soohak: A Mathematician-Curated Benchmark for Evaluating Research-level Math Capabilities of LLMs
May 9, 2026Guijin Son, Seungone Kim, Catherine Arnett +73Research-Level Mathematics Olympiad-Level Problem
OptProver: Bridging Olympiad and Optimization through Continual Training in Formal Theorem Proving
Apr 26, 2026Chenyi Li, Yanchen Nie, Zhenyu Ming +3Theorem Proving Olympiad-Level Problem
School of Mathematical Sciences, Peking University · 2Huawei Technologies · Center for Machine Learning Research, Peking University +1
OMIBench: Benchmarking Olympiad-Level Multi-Image Reasoning in Large Vision-Language Model
Apr 22, 2026Qiguang Chen, Chengyu Luan, Jiajun Wu +7Multimodal Reasoning Benchmarks Recent Vision-Language Models
Research Center for Social Computing and Interactive Robotics, Harbin Institute of Technology · Central South University · Fudan University +3
MathNet: a Global Multimodal Benchmark for Mathematical Reasoning and Retrieval
Apr 20, 2026Shaden Alshammari, Kevin Wen, Abrar Zainal +5Olympiad-Level Problem Mathematical Reasoning
MIT · HUMAIN · Individual Researcher +1
Riemann-Bench: A Benchmark for Moonshot Mathematics
Apr 8, 2026Suhaas Garre, Erik Knutsen, Sushant Mehta +1Research-Level Mathematics Olympiad-Level Problem
Surge AI

Sep 27, 2026, cs.AIJ/K move · Enter open · S save

Learning Strategies to Break Judges

Guruprerana Shabadi, Aaditya Naik, Rajeev Alur, Mayur Naik

University of Pennsylvania · Oracle

Judges Agentic Evaluations Adversarial Evaluation Olympiad-Level Problem Learn

Open paper AskPDF arXiv

As AI agents surpass human performance, it becomes exceedingly hard for system designers to evaluate them directly and understand their failure modes. Consequently, agents themselves are being deployed extensively to evaluate, judge, and provide feedback on model traces. But this raises an important question: how can we trust the judge? In this work, we propose an agent-guided method to find weaknesses of agentic judges that expose interpretable failure mechanisms. Our method focuses on mathematical reasoning and proceeds in two stages: first, we deploy adversarial agents to mutate a set of sound proofs by introducing errors, attempting to misguide judges---in other words, injecting errors that judges are unable to catch. Then, we distill these attempts into a small set of mutation strategies which allow us to analyze the failure modes of the judges. To ensure that these strategies are not overfit to the initial set of proofs, we evaluate them by applying the mutation strategies to a held-out set of proofs and querying the same judge. We deploy our method on GPT-5.6-sol and Claude Opus 5, paired with their agent orchestrators, Codex and Claude Code, respectively. These are used both as mutators to introduce errors and as judges to evaluate correctness of mathematical reasoning. We find that across all the agentic judges, we are able to distill mutation strategies that consistently bypass their evaluations, thereby enabling us to ascertain actionable failure modes. Our analysis also reveals that judge reliability degrades at the frontier: errors in Olympiad-level proofs or graduate-level mathematical texts are detected more consistently, whereas flaws in research-level manuscripts are more likely to escape detection.

Momentum

Related topics

Latest papers 17