cs.CENov 16, 2026

Bookkeeping Is Not Judgement: EJBench, an Engineering Judgement Benchmark Built from a Conjugate Heat Transfer Design Campaign

Authors: James Sadler, Rotimi Alabi, Kotub Uddin

Organizations: Gradient Dynamics, London, UK · AESC, London, UK

Abstract

We introduce EJBench, a benchmark of engineering judgement: whether a model can predict how a design behaves at new conditions from a few earlier results, and decide on it. Unlike engineering benchmarks that test whether an agent can operate simulation or CAD software, EJBench is software-agnostic and asks what an engineer would conclude without running a simulation. Its tasks come from a conjugate heat transfer design campaign for battery cold plates, in three families: matching unlabelled test results to their operating points, choosing which stored runs to reveal before signing off a design, and ranking new duties by peak cell temperature. Every answer is taken from a converged simulation, not from an LLM judge. The best of ten frontier models, GPT-6 Astra, scores 0.71, and models state almost the same confidence on tasks they get wrong as on tasks they get right. We also post-train Qwen3.5-9B with LoRA and reinforcement learning, rewarded by the share of answer criteria the simulation results confirm; on the held-out test split its score rises from 0.071 to 0.106. We release the train, validation and test splits, the evaluation harness and the leaderboard.

Figures & tables

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. PLE: Physical-Engineering's Last Exam

    Oct 21, 2026Rujun Han, Yufan Zhuang, Arijit Ray +10AI Agent BenchmarksBenchmark Design

  2. PhysicsBench: A Unified Leaderboard for Generative and Predictive Models in Engineering Design and Simulation

    Aug 25, 2026Sang Won Lee, Hyogu Jeong, Namwoo KangGenerative AI EvaluationBenchmark Design

  3. BenchBench: Benchmarking Automated Benchmark Generation

    Mar 21, 2026Yandan Zheng, Haoran Luo, Zhenghong Lin +2Benchmark DesignSynthetic Benchmark Generation