Synthetic Benchmark Generation

Ask about this topic

Momentum

23 papers in the last four weeks, up 229% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 213

All topics
CardsList

  1. ClassEval-Pro: A Cross-Domain Benchmark for Class-Level Code Generation

    Apr 29, 2026Yeheng Chen, Chaoxiang Xie, Yuling Shi +4Code GenerationSynthetic Benchmark Generation

    Shanghai Jiao Tong University China

  2. BioGraphletQA: Knowledge-Anchored Generation of Complex QA Datasets

    Apr 28, 2026Richard A. A. Jonker, Bárbara Maria Ribeiro de Abreu Martins, Sérgio MatosBiomedical QASynthetic Benchmark Generation

    IEETA, DETI, LASI, University of Aveiro, Aveiro, Portugal · USF Atlântico Norte, Portugal

  3. EOS-Bench: A Comprehensive Benchmark for Earth Observation Satellite Scheduling

    Apr 28, 2026Qian Yin, Jiaxing Li, Jiaqi Cheng +23Benchmark ConstructionBenchmark Design

    School of Traffic and Transportation Engineering, Central South University, Changsha 410083, China · School of Engineering and Materials Science, Queen Mary University of London, London E1 4NS, UK · College of Automation, Central South University, Changsha, 410083, China +20

  4. STELLAR-E: a Synthetic, Tailored, End-to-end LLM Application Rigorous Evaluator

    Apr 27, 2026Alessio Sordo, Lingxiao Du, Meeka-Hanna Lenisa +2LLM EvaluationSynthetic Data Generation

    1Deutsche Bank, Berlin Technology Center, Berlin, Germany

  5. SyMTRS: Benchmark Multi-Task Synthetic Dataset for Depth, Domain Adaptation and Super-Resolution in Aerial Imagery

    Apr 23, 2026Safouane El Ghazouali, Nicola Venturi, Michael Rueegsegger +1Domain AdaptationMonocular Depth Estimation

    TOELT LLC AI lab / HSLU Winterthur, Swintzerland · Competence Center for Artificial Intelligence and Simulation, armasuisse S+T, 3602 Thun, Switzerland

  6. RARE: Redundancy-Aware Retrieval Evaluation Framework for High-Similarity Corpora

    Apr 21, 2026Hanjun Cho, Jay-Yoon LeeSynthetic Benchmark GenerationBenchmark Validity

    Allganize · Seoul National University

  7. The High Explosives and Affected Targets (HEAT) Dataset

    Apr 20, 2026Bryan Kaiser, Kyle Hickmann, Sharmistha Chakrabarti +6Surrogate ModelingSynthetic Benchmark Generation

    Los Alamos National Laboratory

  8. TPS-CalcBench: A Benchmark and Diagnostic Evaluation Framework for LLM Analytical Calculation Competence in Hypersonic Thermal Protection System Engineering

    Apr 20, 2026Jinglai Zheng, Chuhan Qiao, Haiming HuangSynthetic Benchmark GenerationLLM Safety Evaluation

    School of Civil Engineering, Beijing Jiaotong University, Beijing 100044, China

  9. A Benchmark Construction and Evaluation Framework for Specialist Domains: Case Study on Defense-related Documents

    Apr 20, 2026Bao Gia Doan, Aditya Joshi, Pantelis Elinas +4Synthetic Benchmark GenerationRAG Evaluation

    UNSW Sydney, Australia · Cyndr AI, Australia

  10. TransXion: A High-Fidelity Graph Benchmark for Realistic Anti-Money Laundering

    Apr 19, 2026Keyang Chen, Mingxuan Jiang, Yongsheng Zhao +9Financial Fraud DetectionSynthetic Benchmark Generation

    Fudan University · Peking University

  11. Beyond Static Benchmarks: Synthesizing Harmful Content via Persona-based Simulation for Robust Evaluation

    Apr 18, 2026Huije Lee, Jisu Shin, Hoyun Song +2Synthetic Data GenerationSynthetic Benchmark Generation

    Korea Advanced Institute of Science and Technology (KAIST)

  12. Fun-TSG: A Function-Driven Multivariate Time Series Generator with Variable-Level Anomaly Labeling

    Apr 14, 2026Pierre Lotte, André Péninou, Olivier TesteTime Series GenerationAnomaly Localization

    EPE UT, IRIT · Université de Toulouse, IRIT, CNRS · UT2J, IRIT-SIG, IRIT +2

  13. DiscoGen: Procedural Generation of Algorithm Discovery Tasks in Machine Learning

    Mar 18, 2026Alexander D. Goldie, Zilin Wang, Adrian Hayler +17Automated Algorithm DiscoverySynthetic Benchmark Generation

    University of Oxford · University of California, Santa Barbara · University College London +1

  14. Code2Math: Can Your Code Agent Evolve Math Problems Through Exploration?

    Mar 3, 2026Dadi Guo, Yuejin Xie, Qingyu Liu +11Synthetic Benchmark Generation

    Hong Kong University of Science and Technology · Tsinghua University · Zhejiang University +4

  15. SynthSAEBench: Evaluating Sparse Autoencoders on Scalable Realistic Synthetic Data

    Feb 16, 2026David Chanin, Adrià Garriga-AlonsoSparse AutoencodersSynthetic Benchmark Generation

    University College London · 2MATS · 3Decode Research

  16. LogicEnvGen: Task-Logic Driven Generation of Diverse Simulated Environments for Embodied AI

    Jan 20, 2026Jianan Wang, Siyang Zhang, Bin Li +4Robotics SimulationAI Agent Evaluation

    College of Computer Science and Technology, National University of Defense Technology · Intelligent Game and Decision Lab (IGDL), Beijing · School of Artifical Intelligence, Shanghai Jiao Tong University

  17. An Evolutionary Framework for Automatic Optimization Benchmark Generation via Large Language Models

    Jan 19, 2026Yuhiro Ono, Tomohiro Harada, Yukiya MiuraEvolutionary OptimizationBenchmark Design

    Graduate School of Systems Design, Tokyo Metropolitan University, 6-6, Asahigaoka, Hino, 1910065, Tokyo, Japan. · Graduate School of Science and Engineering, Saitama University, 2552026 Shimo-okubo, Sakura-ku, Saitama, 3388570, Saitama, Japan.

  18. SpaRRTa: A Synthetic Benchmark for Evaluating Spatial Intelligence in Visual Foundation Models

    Jan 16, 2026Turhan Can Kargin, Wojciech Jasiński, Adam Pardyl +2Spatial Reasoning BenchmarksVisual Spatial Reasoning

    Faculty of Mathematics and Computer Science, Jagiellonian University, Kraków, Poland. · Doctoral School of Exact and Natural Sciences, Jagiellonian University, Kraków, Poland. · AGH University of Krakow, Kraków, Poland. +3

  19. Kinship Data Benchmark for Multi-hop Reasoning

    Jan 12, 2026Tianda Sun, Dimitar KazakovLLM EvaluationSynthetic Benchmark Generation

    Department of Computer Science University of York, UK

  20. Genie Sim 3.0 : A High-Fidelity Comprehensive Simulation Platform for Humanoid Robot

    Jan 5, 2026Chenghao Yin, Da Huang, Di Yang +16Robotic Data CollectionRobotics Simulation

  21. Pistachio: Towards Synthetic, Balanced, and Long-Form Video Anomaly Benchmarks

    Nov 22, 2025Jie Li, Hongyi Cai, Mingkang Dong +4Synthetic Benchmark GenerationVideo Anomaly Detection

    Shanghai Jiao Tong University · University of Science & Technology Beijing · Universiti Malaya +2

  22. Ground-Truth Subgraphs for Better Training and Evaluation of Knowledge Graph Augmented LLMs

    Nov 6, 2025Alberto Cattaneo, Carlo Luschi, Daniel JustusGraph Retrieval-Augmented GenerationSynthetic Benchmark Generation

    Graphcore Research

  23. SafetyFlow: An Agent-Flow System for Automated LLM Safety Benchmarking

    Aug 21, 2025Xiangyang Zhu, Yuan Tian, Chunyi Li +3LLM Safety BenchmarksSynthetic Benchmark Generation

    Shanghai AI Lab · East China Normal University

  24. Instance Generation for Patient-to-room Assignment and Admission Scheduling Based on Real Hospital Data

    Jul 4, 2025Tabea Brandt, Christina Büsing, Johanna Leweke +2HealthcareDynamic Programming

    Chair of Combinatorial Optimization, RWTH Aachen University, Im Süsterfeld 9, 52072 Aachen, Germany · Chair of Algorithms and Complexity, RWTH Aachen University, Ahornstraße 55, 52074 Aachen, Germany

  25. ToxSyn-PT: A Synthetic Fine-Grained Dataset of Minority-Targeted Toxic Language in Portuguese

    Jun 11, 2025Iago Alves Brito, Julia Soares Dollis, Fernanda Bufon Farber +2Synthetic Data GenerationHate Speech Detection

    Advanced Knowledge Center for Immersive Technologies (AKCIT) · Federal University of Goiás

  26. PBEBench: A Multi-Step Programming by Examples Reasoning Benchmark inspired by Historical Linguistics

    May 29, 2025Atharva Naik, Prakam, Yash Mathur +6Logical ReasoningLLM Evaluation

    Carnegie Mellon University · Independent Researcher · Ohio State University

  27. Synthetic Enclosed Echoes: A New Dataset to Mitigate the Gap Between Simulated and Real-World Sonar Data

    May 21, 2025Guilherme de Oliveira, Matheus M. dos Santos, Paulo L. J. Drews-JrSynthetic-to-Real Domain AdaptationUnderwater Imaging

    NAUTEC - Intelligent robotics and automation group, FURG - Federal University of Rio Grande, Rio Grande, RS, Brazil · ICMC - Institute of Mathematics and Computer Sciences, USP - University of S˜ao Paulo, S˜ao Carlos, SP, Brazil

  28. Graph Alignment for Benchmarking Graph Neural Networks and Learning Positional Encodings

    May 19, 2025Adrien Lagesse, Marc LelargeGraph Neural NetworksSynthetic Benchmark Generation

    INRIA, École Normale Supérieure - PSL, Paris, France.

  29. AneumoBench: A Source-Linked Benchmark for Synthetic-Geometry Transfer in Aneurysm CFD

    May 19, 2025Xigui Li, Yuanye Zhou, Feiyang Xiao +16Neural Surrogate ModelingSynthetic Data Pretraining

    Artificial Intelligence Innovation and Incubation Institute, Fudan University, Shanghai, China · Shanghai Academy of Artificial Intelligence for Science, Shanghai, China · Human Phenome Institute, Fudan University, Shanghai, China +1

  30. Benchmarking LLMs' Mathematical Reasoning with Unseen Random Variables Questions

    Jan 20, 2025Zijin Hong, Hao Wu, Su Dong +8Mathematical Reasoning BenchmarksSynthetic Benchmark Generation

    The Hong Kong Polytechnic University, Hong Kong SAR, China · University of Electronic Science and Technology of China, Chengdu, China · Tencent Youtu Lab, Shanghai, China +2