Post-Training Large Language Models with LLMON

Basel Shbita; Michael Hind; Chad DeLuca; Nathan Fulton; Shubhi Asthana; Farhan Ahmed; Anisa Halimi; Yada Zhu

cs.AIOct 28, 2026

Post-Training Large Language Models with LLMON

Authors: Basel Shbita, Michael Hind, Chad DeLuca, Nathan Fulton, Shubhi Asthana, Farhan Ahmed, Anisa Halimi, Yada Zhu,

Organizations: IBM Research, San Jose, CA, USA · IBM Research, Yorktown Heights, NY, USA · IBM Research, Cambridge, MA, USA · IBM Research, Dublin, Ireland

Abstract

Large Language Models (LLMs) treat all context tokens uniformly, lacking explicit separation between executable instructions and passive data. This leads to prompt injection vulnerabilities, ambiguous instruction binding, and unreliable execution control. We present a post-training methodology using LLMON (LLM Object Notation), a framework that enables separating instructions from data and requires explicit identifier-based execution. We construct a large-scale structured training corpus of 4.6B tokens spanning general, RAG (Retrieval-Augmented Generation), and tool-calling data, augmented with distractor-infused variants in which competing instructions and irrelevant data segments are present in context but not referenced. Post-training two Granite-4.1 base models (3B and 8B) through supervised fine-tuning followed by reinforcement learning, we show that LLMON-structured supervision reduces attack success rates on indirect prompt-injection benchmarks from 53-100% to 0.02-5.6% while preserving general capabilities. We further introduce attention-based interpretability metrics that diagnose failure modes at individual transformer layers for structured inputs, revealing how our approach reshapes internal attention patterns. Our results demonstrate that explicit execution semantics can be taught through standard post-training without architectural modification.

Figures & tables

Model	Training	General	Alignment	Math	Code	Tool-Calling	Multilingual	Safety
Granite-4.1-3B	Instruct	57.91	82.30	75.19	64.19	60.80	59.89	80.89
Granite-4.1-3B	LLMON	57.16	81.02	73.25	65.45	60.71	62.68	89.64
Granite-4.1-8B	Instruct	64.95	87.06	84.09	72.38	68.27	68.68	84.19
Granite-4.1-8B	LLMON	63.31	86.69	79.63	71.01	67.93	68.24	90.36

Table 1: LLMON-structured post-training vs. Instruct chat-template baselines under matched full fine-tuning. Each column is the unweighted mean over the benchmarks of that category (Section 4.3.1 ). Capability is broadly preserved (the six-category capability mean is unchanged at 3B and 1.4 points lower at 8B) while Safety improves at both scales. Best per model/category in bold .

Model	Training	OPI $\downarrow$	ASB $\downarrow$
Granite-4.1-3B	Instruct	100.0	99.2
Granite-4.1-3B	LLMON	0.1	5.6
Granite-4.1-8B	Instruct	96.3	52.8
Granite-4.1-8B	LLMON	0.02	3.1

Table 2: LLMON-structured post-training vs. Instruct chat-template baselines, measured by attack success rate (ASR, %) on two published indirect prompt-injection benchmarks: OPI (text-level injection) and ASB (agentic tool-observation injection). Lower is better : ASR is the fraction of adversarial prompts that succeed, so a smaller value means a more robust model. Best (lowest ASR) per model/benchmark in bold .

Model	Training	Indirect	Direct	Multi-turn
Granite-4.1-3B	Instruct	2.8	17.6	15.0
Granite-4.1-3B	LLMON	84.6	37.1	28.0
Granite-4.1-8B	Instruct	0.7	9.2	15.0
Granite-4.1-8B	LLMON	79.8	32.7	37.0

Table 3: Adversarial red-teaming with ARES, reported as robustness (

1-\text{ASR}

, %) , higher is better , the opposite direction to Table 2 . Indirect attacks use StruQ-style payloads planted in input, averaged over in-distribution and out-of-distribution splits (ID: 90.3/83.6, OOD: 79.0/75.9 for 3B/8B); direct attacks place the adversarial instruction in the user turn; multi-turn is a Crescendo variant with a 20-turn budget. Best per model/column in bold .

	IFR $\uparrow$		DAS $\uparrow$		LEAK $\downarrow$
Layer	LLMON	Inst.	LLMON	Inst.	LLMON	Inst.
25	86.5	25.8	59.1	38.0	8.6	44.9
28	87.2	37.7	62.2	36.6	6.3	34.9
31	63.8	15.3	53.9	37.2	12.2	40.6
34	75.8	26.3	57.9	35.6	6.4	47.2
39	28.0	17.2	47.0	40.2	13.7	39.7

Table 4: Attention metrics (%) averaged over the distractor examples, at five representative layers, for Granite-4.1-3B LLMON-trained vs. Instruct . The LLMON model holds higher instruction focus and lower leakage at every layer measured; the gap is widest at layers 25-34 and narrows at the output.

Figure 2: Instruction Focus Ratio (IFR) per transformer layer for the LLMON-trained Granite-4.1-3B (teal) vs. its Instruct counterpart (pink) on a distractor example. The two are indistinguishable through layer 20 and separate sharply from layer 22, with the LLMON model sustaining 55-89% across layers 25-38. Both fall at the final layer.

Figure 3: Distractor leakage (LEAK) per transformer layer. Both models leak heavily in the first six layers; from layer 7 the LLMON-trained model (teal) settles into a 5-25% band while the Instruct model (pink) remains between 20% and 50% and rises again in the late layers.

Figure 4: Attention metrics and per-span attention distribution at layer 31 for the LLMON-trained Granite-4.1-3B (left) vs. its Instruct counterpart (right). The LLMON model divides its attention between the bound target instruction (I-T) and target data (D-T); the Instruct model’s largest single recipient is a distractor instruction (I-D1, 47.5%).

Figure 5: Data Attention Score (DAS) per transformer layer for the LLMON-trained Granite-4.1-3B (teal) vs. its Instruct counterpart (pink). DAS separates in the same region of the stack as IFR and LEAK, rising into a 46-94% band across layers 23-38 for the LLMON model.

Figure 6: Attention correlation weights between each generated token (columns, in decoding order) and the KV-cache spans (rows) for the LLMON-trained Granite-4.1-3B at layer 31. Brighter cells indicate stronger attention. The target data span (D-T) activates while the model quotes the sentence it was asked to count, and the distractor spans (I-D1, I-D2, D-D1) remain near zero throughout decoding.

LLM Post-Training

Large language model post-training adapts pretrained models into more useful, reliable, and specialized assistants through supervised and preference-based learning.

See all 249 papers

LLMON: An LLM-native Markup Language to Leverage Structure and Semantics at the LLM Interface
Mar 23, 2026Michael Hind, Basel Shbita, Bo Wu +5Structured Prompting LLM Prompting
IBM Research Yorktown Heights, New York · IBM Research San Jose, California · IBM Research Cambridge, Massachusetts
Analysis of LLMs Against Prompt Injection and Jailbreak Attacks
Feb 24, 2026Piyush Jaiswal, Aaditya Pratap, Shreyansh Saraswati +2Prompt Injection Attacks on LLMs Language Model Safety Evaluation
NIT Trichy, Tiruchirappalli, India · NCE Chandi, Nalanda, India · Bishop Cotton Boys’ School, Bengaluru, India +2
UniGuardian: A Unified Defense for Detecting Prompt Injection, Backdoor Attacks and Adversarial Attacks in Large Language Models
Feb 18, 2025Huawei Lin, Yingjie Lao, Tony Geng +2LLM Backdoor Attacks Adversarial Attacks on LLMs
Rochester Institute of Technology · Tufts University · University of Rochester +1

Mar 23, 2026J/K move · Enter open · S save

LLMON: An LLM-native Markup Language to Leverage Structure and Semantics at the LLM Interface

Michael Hind, Basel Shbita, Bo Wu, Farhan Ahmed+4

IBM Research Yorktown Heights, New York · IBM Research San Jose, California · IBM Research Cambridge, Massachusetts

Structured Prompting LLM Prompting LLM Safety

Open paper AskPDF arXiv

Textual Large Language Models (LLMs) provide a simple and familiar interface: a string of text is used for both input and output. However, the information conveyed to an LLM often has a richer structure and semantics, which is not conveyed in a string. For example, most prompts contain both instructions ("Summarize this paper into a paragraph") and data (the paper to summarize), but these are usually not distinguished when passed to the model. This can lead to model confusion and security risks, such as prompt injection attacks. This work addresses this shortcoming by introducing an LLM-native mark-up language, LLMON (LLM Object Notation, pronounced "Lemon"), that enables the structure and semantic metadata of the text to be communicated in a natural way to an LLM. This information can then be used during model training, model prompting, and inference implementation, leading to improvements in model accuracy, safety, and security. This is analogous to how programming language types can be used for many purposes, such as static checking, code generation, dynamic checking, and IDE highlighting. We discuss the general design requirements of an LLM-native markup language, introduce the LLMON markup language and show how it meets these design requirements, describe how the information contained in a LLMON artifact can benefit model training and inference implementation, and provide some preliminary empirical evidence of its value for both of these use cases. We also discuss broader issues and research opportunities that are enabled with an LLM-native approach.

Abstract

LLM Post-Training

Explore similar work

LLMON: An LLM-native Markup Language to Leverage Structure and Semantics at the LLM Interface

Analysis of LLMs Against Prompt Injection and Jailbreak Attacks

UniGuardian: A Unified Defense for Detecting Prompt Injection, Backdoor Attacks and Adversarial Attacks in Large Language Models