cs.SEMar 20, 2026

An Industrial-Scale Retrieval-Augmented Generation Framework for Requirements Engineering: Empirical Evaluation with Automotive Manufacturing Data

Authors: Muhammad Khalid, Yilmaz Uygun

Organizations: Constructor University Bremen, Bremen, Germany

Abstract

Requirements engineering in Industry 4.0 faces critical challenges with heterogeneous, unstructured documentation spanning technical specifications, supplier lists, and compliance standards. While retrieval-augmented generation (RAG) shows promise for knowledge-intensive tasks, no prior work has evaluated RAG on authentic industrial RE workflows using comprehensive production-grade performance metrics. This paper presents a comprehensive empirical evaluation of RAG for industrial requirements engineering automation using authentic automotive manufacturing documentation comprising 669 requirements across four specification standards (MBN 9666-1, MBN 9666-2, BQF 9666-5, MBN 9666-9) spanning 2015-2023, plus 49 supplier qualifications with extensive supporting documentation. Through controlled comparisons with BERT-based and ungrounded LLM approaches, the framework achieves 98.2% extraction accuracy with complete traceability, outperforming baselines by 24.4% and 19.6%, respectively. Hybrid semantic-lexical retrieval achieves MRR of 0.847. Expert quality assessment averaged 4.32/5.0 across five dimensions. The evaluation demonstrates 83% reduction in manual analysis time and 47% cost savings through multi-provider LLM orchestration. Ablation studies quantify individual component contributions. Longitudinal analysis reveals a 55% reduction in requirement volume coupled with 1,800% increase in IT security focus, identifying 10 legacy suppliers (20.4%) requiring requalification, representing potential $2.3M in avoided contract penalties.

Figures & tables

Explore similar work

CardsList
  1. Agent4RE: A Self-Refining Multi-agent Framework for End-to-End Software Requirements Engineering and Benchmarking

    Oct 7, 2026Yongjian Tang, Linhan Li, Thomas RunklerRequirements EngineeringSoftware Engineering Agents

  2. ReqFusion: A Multi-Provider Framework for Automated PEGS Analysis Across Software Domains

    Mar 24, 2026Muhammad Khalid, Manuel Oriol, Yilmaz UygunRequirements EngineeringAutomated Software Engineering

  3. AGO AI Quality Gate: Evidence-First Release Decisions for Retrieval-Augmented Generation

    Oct 1, 2026Giulio Zeloni, Enrico Lo Conte, Salvatore Rionero +3LLM-as-a-JudgeRAG Evaluation