Machine-Readable Operator Manuals over the ISO 11783 Network for Offline, Multilingual LLM Assistance

Timo Oksanen; Julius Gun

cs.CLOct 22, 2026

Machine-Readable Operator Manuals over the ISO 11783 Network for Offline, Multilingual LLM Assistance

Authors: Timo Oksanen, Julius Gun

Organizations: Chair of Agrimechatronics, Technical University of Munich, Germany · Munich Institute of Robotics and Machine Intelligence (MIRMI), Germany

Abstract

If agricultural machinery is to support intelligent, offline assistance, then the system must access technical documentation dynamically. In this paper, the Semantic Synchronization of operator manuals over ISO 11783-based CAN buses is analyzed, defined as the process that brings the local knowledge base of a tractor-side assistant into agreement with the operator manual of the connected implement. Data transfer performance was benchmarked using Markdown, JSON, and XML; various open-weight Large Language Models (LLMs) were also evaluated in cross-lingual query tasks. The automatic evaluation of the answers was validated against four human raters. A complete manual was transferred over a physical ISO 11783 network in 31 to 89 s, depending on format and transport settings, which is comparable to the object pool upload that operators already wait for at the first coupling of an implement. Markdown produced the smallest files, while the question-answering performance was comparable across the three formats for most models. With retrieval-augmented generation, locally run models answered most questions on the English manual correctly, with no advantage of the largest model over 7B and 8B models, but performed worse on the five other language editions. Keyword retrieval failed across languages, whereas semantic retrieval provided the cross-lingual capability. The exchange requires only a new parameter group that uses the existing request and transport mechanisms of ISO 11783, which makes it a candidate for standardization. The study is a proof of concept with inference on PC-grade hardware; deployment on embedded agricultural hardware remains future work.

Figures & tables

Figure 1: Conceptual architecture for offline Semantic Synchronization.

Figure 2: Representative image from the operator manual (page 86) illustrating PTO speeds for boundary spreading.

Model	Size	Ollama tag	Parameters	Quantization
GPT-OSS	120B	gpt-oss:120b	116.8B	MXFP4
GPT-OSS (judge)	20B	gpt-oss:20b	20.9B	MXFP4
Gemma 3	12B	gemma3:12b	12.2B	Q4_K_M
Qwen 3	8B	qwen3:8b	8.2B	Q4_K_M
DeepSeek-R1	8B	deepseek-r1:8b	8.2B	Q4_K_M
Qwen 2.5	7B	qwen2.5:latest	7.6B	Q4_K_M

Table 1: Overview of LLMs evaluated in this study.

Markdown	JSON	XML
⬇ ## Machine identification In order to provide you with good service various details about your machine may sometimes be required . Please complete these details here in order to have them at hand when they are requested .	⬇ { "page_number" : 2 , "items" : [ { "type" : "SectionHeaderItem" , "text" : "Machine ␣ identification" } , { "type" : "TextItem" , "text" : "In ␣ order ␣ to ␣ provide ␣ you ␣ with ␣ good ␣ service ␣ various ␣ details ␣ about ␣ your ␣ machine ␣ may ␣ sometimes ␣ be ␣ required. ␣ Please ␣ complete ␣ these ␣ details ␣ here ␣ in ␣ order ␣ to ␣ have ␣ them ␣ at ␣ hand ␣ when ␣ they ␣ are ␣ requested." } ,	⬇ <page number= "2" > <text> Machine identification </text> <text> In order to provide you with good service various details about your machine may sometimes be required . Please complete these details here in order to have them at hand when they are requested . </text>

Table 2: Example of different formatting of text on page 2.

Figure 3: Comparison of measured data transfer time over ISOBUS for different manual formats for the English manual (CTS 32).

Figure 4: Token efficiency comparison: XML vs JSON vs Markdown across different languages.

Figure 5: F1 score by model and format: a) English edition, b) five other editions pooled.

Figure 6: F1 score by model and language edition for the three formats.

Figure 7: F1 score for the English edition and for the five other editions pooled, by model and format; points below the diagonal indicate lower scores for the other editions.

Figure 8: Conceptual mockup of the proposed verification interface. The interface displays the generated answer (left) alongside the original source text (right) for verification by the operator.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Format	English edition	Other editions (pooled)
DeepSeek-R1 1.5B	Markdown	0.54 [0.45, 0.62]	0.37 [0.29, 0.45]
	JSON	0.61 [0.51, 0.70]	0.32 [0.24, 0.39]
	XML	0.66 [0.58, 0.73]	0.32 [0.24, 0.39]
Llama 3.2 3B	Markdown	0.77 [0.68, 0.85]	0.47 [0.36, 0.57]
	JSON	0.77 [0.69, 0.85]	0.46 [0.35, 0.56]
	XML	0.72 [0.62, 0.80]	0.50 [0.39, 0.59]

Appendix

Table 3: F1 score [95% confidence interval] by model and format.

	Answerable questions				Unanswerable questions
Model	Correct	Wrong answer	False refusal	Language of manual	Correct abstention	Hallucination
DeepSeek-R1 1.5B	367	420	161	45	410	541
Llama 3.2 3B	419	104	340	160	784	137
Qwen 2.5 7B	545	123	190	147	806	133
Llama 3.1 8B	521	158	118	248	702	197
DeepSeek-R1 8B	557	232	140	59	756	200

Appendix

Table 4: Outcome categories per model (number of answers; 1944 per model, 972 each to answerable and unanswerable questions). Answers in the language of the operator manual are counted for both question types.

Filename	Size (B)	DT Pkts	CM Pkts	Load	Time	Param
dutch_manual.md	319540	45649	1 + 2855	41.1%	66.6 s	CTS 16
dutch_manual.md	319540	45649	1 + 1428	55.6%	47.1 s	CTS 32
dutch_manual.md	319540	45649	1 + 715	70.9%	35.4 s	CTS 64
dutch_manual.md	319540	45649	1 + 358	79.1%	30.8 s	CTS 128
dutch_manual.xml	334470	47782	1 + 1495	55.5%	48.9 s	CTS 32
dutch_manual.json	555565	79367	1 + 2482	54.8%	84.2 s	CTS 32

Appendix

Table 5: Detailed ETP data transfer results comparing languages, formats, and protocol settings (CTS). "DT" refers to Data Transfer packets; "CM" refers to Connection Management packets.

Figure 9: Token and character counts of the converted manuals by format and language edition (tokens: Qwen3-Embedding-8B tokenizer).

	Markdown			JSON			XML
Edition	BM25	Semantic	Hybrid	BM25	Semantic	Hybrid	BM25	Semantic	Hybrid
English	0.89	0.93	0.89	0.91	0.91	0.94	0.80	0.91	0.89
German	0.11	0.87	0.78	0.13	0.91	0.78	0.11	0.89	0.78
French	0.24	0.83	0.80	0.19	0.89	0.83	0.19	0.87	0.74
Dutch	0.20	0.83	0.78	0.19	0.83	0.80	0.17	0.93	0.85
Italian	0.09	0.83	0.80	0.09	0.87	0.78	0.09	0.83	0.78

Appendix

Table 6: Recall@3 of the gold page for the 54 answerable questions (English questions on each language edition).

Model	No context	BM25	Semantic	Hybrid	Oracle
English edition
Llama 3.2 3B	0.11 [0.03, 0.21]	0.74 [0.64, 0.83]	0.75 [0.67, 0.84]	0.76 [0.66, 0.84]	0.80 [0.72, 0.88]
Qwen 2.5 7B	0.00 [0.00, 0.00]	0.81 [0.72, 0.88]	0.81 [0.73, 0.89]	0.81 [0.73, 0.88]	0.87 [0.81, 0.93]
DeepSeek-R1 8B	0.00 [0.00, 0.00]	0.73 [0.63, 0.82]	0.76 [0.67, 0.85]	0.77 [0.67, 0.85]	0.84 [0.76, 0.91]
Gemma 3 12B	0.00 [0.00, 0.00]	0.78 [0.69, 0.85]	0.82 [0.75, 0.89]	0.78 [0.71, 0.86]	0.84 [0.78, 0.90]
Other five editions (pooled)

Appendix

Table 7: F1 score [95% confidence interval] by context condition (Markdown editions; other languages pooled).

	Human majority: match	Human majority: no match
Judge: yes	102	3
Judge: no	10	86

Appendix

Table 8: Verdicts of the judge compared with the majority of four human raters (201 items with a majority).

Data Integration

See all 16 papers

Oct 26, 2026

INVAR-Agri: Detecting Discrepancies in Multilingual Operator Manuals of Agricultural Implements with a Fine-Tuned LLM

Operator manuals of agricultural implements are delivered in many languages, and the standards require that competent persons review every translation, which makes this review the bottleneck. This paper presents a proof of concept that compares two training workflows on the same LLM, EuroLLM-1.7B-Instruct, trained on operator manuals of one manufacturer. In approach A, a back-translator translates each language version back into English, and an English reader compares it with the English version. In approach B, a checker reads the English version and the translation together, where the checker reports differences. The test sets contain errors planted in numbers, units, opposite words, obligations, negations, and conditions in 24 languages, and they include unmodified text and meaning-preserving edits for alerts and false alarms. The trained back-translator concealed many planted errors, most of all weakened obligations, while translation-quality scores rose. In contrast, the separately trained checker detected most planted errors, although it was weak for units and for number changes unlike those in training. A check by the author confirmed the concealment. In addition, a simple rule check caught the number and unit changes, but it alerted on harmless rewrites. A check of alerts on real Finnish translations showed that most alerts were false, caused by language-specific wording, so the precision on real text remains a limitation. Within this setting, a trained checker might support, not replace, the review by competent persons, and firmer conclusions need more data.

Timo Oksanen

Technical University of Munich, Germany; Chair of Agrimechatronics; Munich Institute of Robotics and Machine Intelligence (MIRMI)

Language Model Error Detection Document Understanding Multilingual Language Model Evaluation Back-Translation

Ask PDF

Sep 18, 2026

Offline Multimodal Large Language Models for Decision Support in Air Operations

Air operations rely on complex rules, established procedures, and time-critical analysis under limited connectivity and strict security constraints. In such environments, analysts must combine written doctrine with images, often without access to external computing resources. This paper studies offline large language models as decision support tools, deployed in isolated and restricted environments to give analysts access to doctrinal knowledge that remains traceable to its original sources through natural language interaction. We describe a modular retrieval-augmented architecture suitable for operation without Internet connectivity, supporting both text and image input from technical manuals. As a first step toward evaluating this architecture, we report a pilot study with four image analysts of the Brazilian Air Force, combining (i) a doctrinal knowledge assessment based on their electronic-target identification doctrine, comparing human and proposed system performance on the same test, and (ii) a measurement of the cognitive workload involved in manually producing a reconnaissance target report (Relatório de Missão de Reconhecimento - REMIR) without AI assistance. The results show a demanding manual task, especially in terms of mental demand (6.0/7) and effort (5.0/7), while the proposed system matches the human score (8/10) and completes the assessment in 7.1 minutes (compared to a human average of 26.5 minutes), establishing a baseline for future AI-assisted evaluation. Finally, we describe a future evaluation protocol to systematically compare manual and AI-assisted workflows.

Joao P. A. Dantas, Jelton A. Cunha, Gabriel Dietzsch

Instituto de Estudos Avançados, São José dos Campos, São Paulo, Brazil, 12.228-001

AI-Assisted Decision Making Multimodal Large Language Models Retrieval-Augmented Generation

Ask PDF

Oct 26, 2026

Offline Language-Based Support System for Tactical Decision-Making in Air Combat Operations

An offline language-based support tool is presented to assist pilot training and tactical decision-making in air combat operations. The system, named MATE (Mission Assistant for Training and Evaluation), applies retrieval-augmented generation (RAG) to answer natural language questions using content extracted from doctrinal and tactical manuals. Built entirely with open-source components, it performs text preprocessing, embedding generation, semantic search, and language generation locally, ensuring full offline functionality. The tool was evaluated by three Brazilian Air Force fighter pilots, who rated the answers produced with three open-weight language models (Llama 3.1 8B, Qwen 2.5 14B, and Mistral NeMo 12B) to five tactical questions derived from a publicly available air combat manual. Average ratings ranged from 4.00 to 4.40 on a five-point scale, with Mistral NeMo 12B obtaining the highest score, and the pilots rated the response time of the tool as fully adequate compared to manual search. The relevance of the retrieved content was identified as the main point for improvement. Potential applications are discussed in onboard mission systems, flight simulators, and training environments.

Joao P. A. Dantas

Decision Support Systems Subdivision Institute for Advanced Studies São José dos Campos, São Paulo, Brazil, 12228-001

AI-Assisted Decision Making Language Model Generation Evaluation

Ask PDF

Abstract

Data Integration

Explore similar work

INVAR-Agri: Detecting Discrepancies in Multilingual Operator Manuals of Agricultural Implements with a Fine-Tuned LLM

Offline Multimodal Large Language Models for Decision Support in Air Operations

Offline Language-Based Support System for Tactical Decision-Making in Air Combat Operations