cs.CLFeb 27, 2026

Full-Stack Domain Enhancement for Combustion LLMs: Construction and Optimization

Authors: Quanjia Xiao, Weimin Ouyang, Zonglin Yang, Tianhao Wu, Qingguo Zhou, Runze Mao, Zhi X. Chen

Organizations: Peking University · AI for Science Institute, Beijing · DP Technology

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Large language models (LLMs) in the direction of task adaptation and capability enhancement for professional fields demonstrate significant application potential. Nevertheless, for complex physical systems such as combustion science, general-purpose LLMs often generate severe hallucinations due to insufficient domain knowledge and the inability to adhere to physical conservation laws. To address this issue, we propose the first full-stack domain-enhanced LLM workflow tailored for the field of combustion science, which integrates automated domain corpus construction, incremental pre-training, instruction fine-tuning, and verifiable reward-based reinforcement learning. This workflow ensures that the model truly internalizes physical laws rather than merely learning textual statistical patterns. We also release FlameBench, a standardized evaluation benchmark specifically designed for complex reasoning tasks in combustion science. Experimental results demonstrate that the model developed in this work significantly outperforms state-of-the-art general-purpose closed-source models and traditional retrieval-augmented generation methods on combustion science reasoning tasks. This work lays a solid technical and resource foundation for the subsequent development of domain-specific scientific research agents with reliable scientific reasoning capabilities.

Figures & tables

Figure 1: Dataset token distribution by category.
Figure 2: Data processing pipeline for the combustion-specific pre-training corpus.
Figure 3: The construction pipeline of FlameBench.
Figure 4: Architecture of the Closed-loop RAG Assessment System.
Figure 5: Loss and learning rate curves during the Continue Pre-training and SFT stages.
Model GroupAccuracy (%)
Qwen3-8B-Base26.8
CPT33.3
SFT-General33.5
SFT-Combustion35.1
RLVR-Opt43.8
Table 1: Performance Comparison of Multi-stage Training
ModelAccuracy (%)
GPT-515.60
GLM-432.64
Gemini Pro32.10
DeepSeek-R128.37
Average Score27.18
Table 2: Performance Comparison of Different Models
MethodAccuracy (%)
RAG + GPT-516.52
RAG + GLM-432.09
RAG + Gemini Pro27.80
RAG + DeepSeek-R128.54
Average Score26.24
Table 3: Performance of RAG with Different Models
Figure 6: Comparison of training metrics between SFT-Combustion RL and Qwen-8B RL.
Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

HyperparameterCPTSFT-GeneralSFT-Combustion
Learning Rate2.0e-55.0e-52.0e-5
LR SchedulerWSDCosineCosine
Max Length16,38416,38420,000
Batch Size (Total)256256128
Precisionbf16bf16bf16
Warmup Ratio0.010.050.03
Appendix
Table 4: Hyperparameters for CPT and SFT Stages.
ParameterValue
AlgorithmGRPO
Actor Learning Rate2.0×10−62.0\times 10^{-6}
Global Training Batch Size128
Max Prompt Length1,024
Max Response Length8,192
KL Coefficient ( β\beta )0.005
Appendix
Table 5: Hyperparameters for the GRPO reinforcement learning stage.
More in this topic

Domain Adaptation for LLMs

Domain adaptation for large language models tailors their knowledge and behavior to specialized fields, languages, and tasks.

See all 94 papers

Explore similar work

CardsList

  1. A unified foundational framework for knowledge injection and evaluation of Large Language Models in Combustion Science

    Feb 27, 2026Zonglin Yang, Runze Mao, Tianhao Wu +3Knowledge Augmentation for Language ModelsLLM Evaluation

    School of Mechanics and Engineering Science, Peking University, Beijing, 100871, PRChina · AI for Science Institute, Beijing, 100080, China

  2. Sci-CoE: Co-evolving Scientific Reasoning LLMs via Geometric Consensus with Sparse Supervision

    Feb 12, 2026Xiaohan He, Shiyang Feng, Songtao Huang +3Scientific Reasoning in Language ModelsLanguage Model Reasoning Evaluation

    Shanghai Artificial Intelligence Laboratory · Fudan University

  3. OMNIFLOW: A Physics-Grounded Multimodal Agent for Generalized Scientific Reasoning

    Mar 16, 2026Hao Wu, Yongheng Zhang, Yuan Gao +7Scientific Reasoning in Language ModelsPhysics-Informed ML

    Tsinghua University · Tencent · Shenzhen Loop Area Institute +2