cs.CLNov 16, 2026

IndustrialKnowEval: A Benchmark for Evaluating Multi-Level Industrial Knowledge in LLMs

Authors: Haiteng Wang, Haoze Li, Jingwei Guo, Jingbo Zhou, Si Liu, Lei Ren

Organizations: School of Artificial Intelligence, Beihang University, Beijing 100191, China · National College for Excellent Engineers, Beihang University, Beijing 100191, China · School of Automation Science and Electrical Engineering, Beihang University, Beijing 100191, China · Business Intelligence Lab, Baidu Research, Beijing 100085, China · 

Abstract

Large language models (LLMs) are increasingly explored for industrial tasks, yet few benchmarks evaluate their mastery of industrial knowledge across different types of cognitive demand. We introduce IndustrialKnowEval (IndKnow), a benchmark for industrial knowledge and engineering question answering. It contains 43,282 samples covering 21 task types across seven industrial sectors, including aerospace, electric power, and metallurgy, organized into four levels of cognitive demand: memory, comprehension, reasoning, and engineering application. Enterprise-based domain specialists and experienced researchers participate in all stages of dataset construction and quality control. We conduct a series of experiments on 18 representative LLMs, with domain experts deeply involved in the evaluation. Building upon IndKnow, we propose the Industrial Cognitive Capability Gap (ICCG) metric to characterize the performance differential between foundational industrial knowledge and higher-order reasoning and application, and we analyze it alongside absolute performance to provide a more complete view of model capabilities. We anticipate that IndustrialKnowEval will support the benchmarking of LLMs in industrial scenarios and promote the development of robust industrial intelligence. Code is available at: https://github.com/J4cksonLee/IndKnow

Figures & tables

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. IndustryCode: A Benchmark for Industry Code Generation

    Apr 3, 2026Puyu Zeng, Zhaoxi Wang, Zhixu Duan +7Code GenerationCode Generation Benchmarks

  2. IndustryBench: Probing the Industrial Knowledge Boundaries of LLMs

    May 11, 2026Songlin Bai, Xintong Wang, Linlin Yu +12LLM EvaluationQuestion Answering

  3. FailureSensorIQ: A Multi-Choice QA Dataset for Understanding Sensor Relationships and Failure Modes

    Jun 3, 2025Christodoulos Constantinides, Dhaval Patel, Shuxin Lin +3Question AnsweringLLM Reliability