Domain Adapted Large Language Models for Additive Manufacturing

Peter Pak; Amir Barati Farimani

cs.LGMar 23, 2026

Domain Adapted Large Language Models for Additive Manufacturing

Authors: Peter Pak, Amir Barati Farimani

Organizations: Department of Mechanical Engineering, Carnegie Mellon University, Pittsburgh, PA, USA · Machine Learning Department, Carnegie Mellon University, Pittsburgh, PA, USA

Abstract

This work presents a collection of multi-modal domain adapted large language models built upon the instruction tuned variants of open weight models (Gemma 3, Qwen 3, Gemma 4) using a relatively small dataset of around 50 million tokens. The dataset consists of open-access additive manufacturing journal articles with data extracted for the domain adaptive pretraining and visual instruction tuning processes. Various stages of the developed model are evaluated with the Additive-Manufacturing-Benchmark which consists of additive manufacturing domain specific tasks compiled published resources. Domain adapted and instruction tuned models exhibit proficiency in both language and vision based tasks, achieving accuracies upwards of 90% in general additive manufacturing knowledge. This domain adaptive pretraining and instruction tuning strategy outline an accessible specialization method for large language models to a domain such as additive manufacturing.

Figures & tables

Figure 1: General training and evaluation process utilized in the development of various additive manufacturing specific large language models. (1) Domain adaptive pretraining (DAPT) is applied in both the text and vision modalities using the Gemma 3 model using a selection of open-access articles from the additive manufacturing domain. Team et al. (2025) as its base. (2) Visual Instruction Tuning (VIT) utilizes the extracted captions from these articles and generates description and conversation prompts with the aid of GPT-OSS 120B OpenAI et al. (2025) . The domain adapted model is evaluated with tasks from the Additive-Manufacturing-Benchmark which are compiled from various published resources Akbari et al. (2022) ; Scime et al. (2023) ; Hu et al. (2024) .

Figure 2: Original figure by Hu et al. Hu et al. (2021) diagrams LoRA for a square pretrained weight matrix ( $d\times d$ ). Figure is adapted to showcase an example of a pretrained weight matrix with different input and output dimensions ( $d\times k$ ).

Figure 3: Composition of each dataset configuration by journal for (left) text, (center) images, and (right) visual instruction tuning (VIT). The reduced Rapid Prototyping Journal slice in the VIT configuration (right) is due to the smaller number of figure and caption pairs found within that journal during the data extraction process.

Figure 4: Content (text) and figures (images) are extracted from their .pdf formats from open-access articles from various additive manufacturing journals. This data is utilized for domain adaptive pretraining and the figure captions are processed into question answer conversations and detailed descriptions using GPT-OSS OpenAI et al. (2025) for visual instruction tuning. The various data configurations are uploaded and publicly accessible via HuggingFace.

Figure 5: Gemma 3 12B variants (pretrained only or instruction tuned) are adapted to the domain of additive manufacturing using LoRA in a 3 step training process. (1) Extracted text from the open-access articles is used to train language model attention weights for next token prediction. (2) Extracted figures along with their respective captions are utilized to train the vision tower attention weights while keeping the language model weights frozen. (3) Visual instruction tuning post training on all attention weights is performed using question-answer pairs along with detailed image descriptions.

Figure 6: Additive-Manufacturing-Benchmark is a set of multi-modal tasks compiled from various published resources Hu et al. (2024) ; Scime et al. (2023) ; Akbari et al. (2022) used to evaluate AM comprehension. Language based tasks include answering general knowledge questions in (a) short answer or (b) multiple choice formats and (c) melt pool dimension prediction given process parameters. Image-based tasks including (d) FDM Defect identification, (e) AM machine identification, and (f) LPBF anomaly identification.

Figure 7: The visual instruction tuned version of Gemma-3-12b-it exhibits the best performance in most image based tasks and closely follow the performance of the base model with only domain adaptation in text for remaining tasks.

Figure 8: Domain adapted Qwen3-vl-8b-it performs comparably on general knowledge tasks however under performs on vision based benchmarking tasks such as anomaly and defect detection.

Figure 9: The domain adapted 31 billion parameter Gemma 4 model achieves the best results for general knowledge based tasks but performs comparably to other investigated models for image based tasks.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 10: Hypothetical comparison of outputs between standard prompting process to that of the chain-of-thought reasoning process.

Figure 11: Plot indicates vocabulary similarity between the various open-access articles categorized by their respective journal used in the dataset. Additive Manufacturing Letters exhibits the highest vocabulary similarity between all journals with the lowest vocabulary similarity seen between Rapid Prototyping Journal and Journal of Additive Manufacturing .

Figure 12: Common n-grams ordered by frequency from each journal with common stop words and journal information filtered out.

Figure 13: Plot of the top 10 keywords obtained from dataset articles omitting those where none are provided and ones where “Additive Manufacturing” is listed.

Figure 14: All models adapted upon the Gemma-3-12b-pt (pretraining only) variant of the base model exhibit comparatively worse performance to their Gemma-3-12b-it variants. Models at the visual instruction tuned achieve the best performance and in some cases outperform their previous training stages by a significant factor.

Explore similar work

Domain Adaptation of Large Language Models for Polymer-Composite Additive Manufacturing Using Retrieval-Augmented Generation and Fine-Tuning
Apr 2, 2026Saiful Islam Sagor, Tania Haghighi, Minhaj Nur Alam +1Domain Adaptation for LLMs LLM Fine-Tuning
Department of Mechanical Engineering and Engineering Science, University of North Carolina at Charlotte, NC 28223, USA. · Department of Electrical and Computer Engineering, University of North Carolina at Charlotte, NC 28223, USA.
FORGE: Fine-grained Multimodal Evaluation for Manufacturing Scenarios
Apr 8, 2026Xiangru Jian, Hao Xu, Wei Pang +13Multimodal Model Evaluation Multimodal Learning
University of Waterloo, Canada · University of Sydney, Australia · Singapore Management University, Singapore +7
DS $^2$ -Instruct: Domain-Specific Data Synthesis for Large Language Models Instruction Tuning
Mar 13, 2026Ruiyao Xu, Noelle I. Samia, Han LiuSynthetic Data Generation for Language Models LLM Fine-Tuning
Northwestern University

Apr 2, 2026J/K move · Enter open · S save

Domain Adaptation of Large Language Models for Polymer-Composite Additive Manufacturing Using Retrieval-Augmented Generation and Fine-Tuning

Saiful Islam Sagor, Tania Haghighi, Minhaj Nur Alam, Erina Baynojir Joyee

Department of Mechanical Engineering and Engineering Science, University of North Carolina at Charlotte, NC 28223, USA. · Department of Electrical and Computer Engineering, University of North Carolina at Charlotte, NC 28223, USA.

Domain Adaptation for LLMs LLM Fine-Tuning Retrieval-Augmented Generation Question Answering Additive Manufacturing

Open paper AskPDF arXiv

General-purpose large language models (LLMs) often struggle to generate reliable responses in specialized engineering domains due to limited domain grounding and insufficient exposure to structured technical knowledge. This study investigates practical strategies for adapting a foundation LLM to the additive manufacturing (AM) domain in order to improve answer accuracy, relevance, and usability for expert-level question answering. AM knowledge is distributed across heterogeneous sources such as academic literature, manufacturer documentation, technical standards, and procedural guides. Although general LLMs demonstrate strong linguistic capabilities, they frequently fail to retrieve and contextualize such domain-specific information. Two common approaches to address this limitation are domain-specific fine-tuning and retrieval-augmented generation (RAG). We construct a curated AM corpus and evaluate three configurations based on LLaMA-3-8B: (1) the pretrained baseline model, (2) a RAG system that retrieves relevant document chunks from a vector database, and (3) a model fine-tuned on raw domain text. Performance is evaluated using 200 expert-designed AM questions assessed by mechanical engineering experts for accuracy, relevance, and overall preference. Results show that the RAG model consistently outperforms the baseline. Among the 200 questions, 75.5% of RAG responses are judged more accurate, 85.2% are preferred overall, and 90.8% are rated more relevant than baseline responses. In contrast, fine-tuning on raw AM text reduces performance, producing more accurate answers in only 5.6% of cases and more relevant answers in 32.5% of cases. These results indicate that retrieval-augmented approaches provide a more effective pathway for adapting LLMs to specialized engineering domains than naive fine-tuning on unstructured technical data.

Abstract

Explore similar work

Domain Adaptation of Large Language Models for Polymer-Composite Additive Manufacturing Using Retrieval-Augmented Generation and Fine-Tuning

FORGE: Fine-grained Multimodal Evaluation for Manufacturing Scenarios

DS2^22-Instruct: Domain-Specific Data Synthesis for Large Language Models Instruction Tuning

DS $^2$ -Instruct: Domain-Specific Data Synthesis for Large Language Models Instruction Tuning