Synthetic Benchmark Generation

Follow this topic Ask about this topic

Momentum

23 papers in the last four weeks, up 229% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 213

All topics

HawkesNest: A Multi-Axis Synthetic Benchmark for Spatiotemporal Pattern Complexity
Jun 15, 2026Yahya Aalaila, Sumantrak Mukherjee, Gerrit Großmann +1Temporal Point Processes Synthetic Benchmark Generation
German Research Center for Artificial Intelligence (DFKI), Data Science and its Applications Research Group, Kaiserslautern, Germany
A Validated LBM Dataset and Pipeline for Surrogate Modeling of Turbulent 3D Obstructed Channel Flows
Jun 15, 2026Lukas Schröder, Shubham Kavane, Harald KöstlerNeural Surrogate Modeling Synthetic Benchmark Generation
Chair of Computer Science 10 (System Simulation) Friedrich-Alexander-Universität Erlangen-Nürnberg
PAL-Bench: Evidence-Grounded Profile Reconstruction from Longitudinal Personal Albums
Jun 15, 2026Qiwei Yan, Zhiqiang Yuan, Zexi Jia +4Data Provenance Synthetic Benchmark Generation
University of Chinese Academy of Sciences, Beijing, China · 1WeChat AI, Tencent, Beijing, China · 3Beihang University, Beijing, China +1
The Data Manifold under the Microscope
Jun 14, 2026Marios Koulakis, Constantin SeiboldIntrinsic Dimensionality Synthetic Benchmark Generation
Independent Researcher, Germany · Diagnostic and Interventional Radiology, University Clinic Heidelberg, Heidelberg, Germany.
Benchmarking Instance-Dependent Label Noise with Controlled Corruptions
Jun 12, 2026Shadman Islam, Agustinus Kristiadi, Mostafa MilaniSynthetic Benchmark Generation Noisy-Label Learning
Department of Computer Science, Western University, London, Ontario, Canada
VHDLSuite: Unified Pipeline for LLM VHDL Generation with Data Synthesis and Evaluation
Jun 11, 2026Yijun Shen, Minghao Shao, Yichen Zhao +4LLM Evaluation Code Generation
Center for Data Science, NYU Shanghai, China · NYU Tandon School of Engineering, USA · NYU Abu Dhabi, UAE
EvoBrowseComp: Benchmarking Search Agents on Evolving Knowledge
Jun 11, 2026Yunhan Wang, Jiaan Wang, Lianzhe Huang +2Benchmark Contamination Web Search Agents
Northeastern University, China · Weixin AI, Tencent Inc, China
SciR: A Controllable Benchmark for Scientific Reasoning in LLMs
Jun 11, 2026Pierre Beckmann, Marco Valentino, Andre FreitasLLM Evaluation Scientific Reasoning
Idiap Research Institute, Switzerland · EPFL, Switzerland · School of Computer Science, University of Sheffield, UK +2
How Fine-Grained Should a RAG Benchmark Be? A Hierarchical Framework for Synthetic Question Generation
Jun 11, 2026Chase M. Fensore, Kaustubh Dhole, Jason Fan +2Benchmark Design Synthetic Data Generation
Department of Computer Science Emory University Atlanta, USA
Constructing Evaluation Datasets for Procedural Reasoning: Balancing Naturalness, Grounding, and Multi-Hop Coverage
Jun 11, 2026Sarah Elshabrawy, Rahul K. Dass, Ashok K. GoelIntelligent Tutoring Systems Synthetic Benchmark Generation
Georgia Institute of Technology, Atlanta GA, USA
Net-Ev $^2$ : A Generative Simulator for Network Event Evolution
Jun 10, 2026Guangyu Wang, Zhaonan WangIntelligent Transportation Systems Synthetic Benchmark Generation
NYU Shanghai Shanghai, China · NYU Shanghai Center for Data Science Shanghai Key Laboratory of Urban Design and Urban Science
Embodied-BenchClaw: An Autonomous Multi-Agent System for Embodied Spatial Intelligence Benchmark Construction
Jun 10, 2026Baoyang Jiang, Fengchun Zhang, Leyuan Wang +7Spatial Reasoning Benchmarks Synthetic Benchmark Generation
QiYuan Lab · School of Information and Software Engineering, University of Electronic Science and Technology of China · Beijing University of Posts and Telecommunications +2
Breaking the Solver Bottleneck: Training Task Generators at the Learnable Frontier
Jun 10, 2026Lorenz Wolf, Connor Watts, Roger Creus Castanyer +4Reinforcement Learning Synthetic Benchmark Generation
Vmax · Goodfire
ClusBench: The Clustering Benchmark Data Resource You've All Been Waiting For (?)
Jun 9, 2026David P. HofmeyrBenchmark Construction Clustering
School of Mathematical Sciences, Lancaster University
STAGE-Claw: Automated State-based Agent Benchmarking for Realistic Scenarios
Jun 9, 2026Sirui Liang, Bohan Yu, Peiyu Wang +8Computer-Use Agent Benchmarks LLM Agent Evaluation
The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation · Chinese Academy of Sciences, Beijing, China, University of Chinese Academy of Sciences · Zhongguancun Academy +3
Internalizing Geometric Law: Learning from Solver Residuals for Precision-Critical Generation
Jun 8, 2026Rafael Cabral, Pang Zixi, Ziyi Shou +1Reward Shaping LLM-Based Program Synthesis
Huawei Celia Team
BSTabDiff: Block-Subunit Diffusion Priors for High-Dimensional Tabular Data Generation
Jun 8, 2026Al Zadid Sultan Bin Habib, Md Younus Ahamed, Prashnna Gyawali +2Copula Models Synthetic Tabular Data Generation
1,2,3,5Lane Department of Computer Science and Electrical Engineering West Virginia University, Morgantown, WV 26506, USA · 4Scientific Computing and Imaging Institute & Department of Biomedical Informatics The University of Utah, Salt Lake City, UT 84112, USA
Declarative Outcome-Conformant Synthesis: Exact, Closed-Form Specification Satisfaction and a Conformance Benchmark
Jun 7, 2026Muhammed RasinSynthetic Tabular Data Generation Synthetic Benchmark Generation
Independent Researcher
PIPE-Cypher: Automatic Enterprise Benchmark Generation for Text-to-Cypher Systems
Jun 7, 2026Suraj Ranganath, Anish RaghavendraBenchmark Design Text-to-Cypher
Halıcıoğlu School of Data Science and Computing University of California, San Diego · Independent Researcher
SurgiQ: A Large-Scale Multi-Domain Benchmark for Evaluating Surgical Understanding in Large Language Models
Jun 6, 2026Ayah Al-Naji, Edoardo Fazzari, Saif Alkindi +3LLM Evaluation Synthetic Benchmark Generation
Mohamed bin Zayed University of Artificial Intelligence, UAE
The CIFAR Synthetic Evidence Corpus for Detecting AI-Generated Evidence
Jun 6, 2026Kelly McConvey, Jalehsadat Mahdavimoghaddam, Nima Jamali +8Digital Forensics Synthetic Benchmark Generation
University of Toronto · University of Waterloo · University of British Columbia +2
A Framework for Evaluating and Benchmarking Concept Drift Detection Methods
Jun 5, 2026Vitor Cerqueira, Heitor Murilo Gomes, Marco Heyden +2Concept Drift Synthetic Benchmark Generation
University of Coimbra · Victoria University of Wellington · Commerzbank +2
UnpredictaBench: A Benchmark for Evaluating Distributional Randomness in LLMs
Jun 4, 2026Amirhossein Abaskohi, Amirhossein Dabiriaghdam, Liang Luo +4LLM Evaluation Language Model Generation Evaluation
University of British Columbia · 2Independent Researcher
Benchmark Everything Everywhere All at Once
Jun 4, 2026Shiyun Xiong, Dongming Wu, Peiwen Sun +5LLM Evaluation Benchmark Design
MMLab, The Chinese University of Hong Kong · CPII under InnoHK · The Chinese University of Hong Kong, Shenzhen +3
PhyRoGen: Synthetic Generation of Physical Robot Manipulation Puzzles Using Procedural Content Generation
Jun 4, 2026Lennart Julian Droß, Andreas Orthey, Marc ToussaintSynthetic Benchmark Generation Robotic Manipulation
Technical University of Berlin, Germany · Robotics Institute Germany
GenTI: Benchmarking LLMs for Autonomous IDPS Rule Generation for Unseen Attacks
Jun 4, 2026Hassan Jalil Hadi, Rehana Yasmin, Ali ShokerNetwork Intrusion Detection Synthetic Benchmark Generation
Cyber Security and Resilience Technology (CyberSaR), King Abdullah University of Science and Technology (KAUST), Saudi Arabia
SANE Schema-aware Natural-language Evaluation of Biological Data
Jun 3, 2026Rolf Gattung, Martin Krueger, Markus ReischlLLM Evaluation LLM Grounding
Institute for Automation and Applied Informatics (IAI), Karlsruhe Institute of Technology (KIT), Hermann-von-Helmholtz-Platz 1, 76344 Eggenstein-Leopoldshafen, Germany.
Measuring What Matters: Synthetic Benchmarks for Concept Bottleneck Models
Jun 3, 2026Julian Skirzynski, Harry Cheon, Shreyas Kadekodi +2Concept Bottleneck Models Benchmark Design
University of California, San Diego
CoEval: Ranking Language Models for Custom Tasks Without Labeled Data or Trustworthy Benchmarks
Jun 2, 2026Alexander Apartsin, Yehudit ApersteinModel Selection LLM Evaluation
Holon Institute of Technology, Israel · Afeka Tel Aviv Academic College of Engineering, Israel
FinStressTS: A Parametric Synthetic Benchmark for Time-Series Forecasting in Finance
Jun 2, 2026Jiaze Sun, Kelvin J. L. Koa, Ruiyang Ni +3Financial Forecasting Time Series Forecasting
National University of Singapore Singapore · National University of Singapore Asian Institute of Digital Finance Singapore · Nanyang Technological University Singapore

Jun 15, 2026, cs.LGJ/K move · Enter open · S save

HawkesNest: A Multi-Axis Synthetic Benchmark for Spatiotemporal Pattern Complexity

Yahya Aalaila, Sumantrak Mukherjee, Gerrit Großmann, Sebastian Vollmer

German Research Center for Artificial Intelligence (DFKI), Data Science and its Applications Research Group, Kaiserslautern, Germany

Temporal Point Processes Synthetic Benchmark Generation

Open paper AskPDF arXiv

Evaluation of spatiotemporal point process (STPP) models relies heavily on opaque real-world datasets, where latent generative structure is unknown and model failures are difficult to attribute. We introduce HawkesNest, a generator-aligned benchmark for controlled spatiotemporal pattern complexity built on a multivariate Hawkes backbone. HawkesNest defines four complexity axes: space--time entanglement, background heterogeneity, cross-type interaction, and domain topology. Each axis is associated with a deterministic index computed from the latent data-generating mechanism. By varying these axes while holding global rate, stability, and simulation budget fixed, HawkesNest enables diagnostic stress tests of STPP models under known structural difficulty. We verify that the indices are monotone and nearly orthogonal under controlled sweeps. We illustrate its use by showing that Hawkes-family baselines degrade under joint heterogeneity--entanglement complexity, even though they are structurally aligned with the Hawkes data-generating backbone. We further show that HawkesNest exposes neural-model sensitivity: AutoSTPP remains vulnerable under isolated increases in space--time entanglement. Code. Available at https://github.com/YahyaAalaila/HawkesNest

Momentum

Related topics

Latest papers 213