LLM Trojans that Increase Response Lengths and Cost

Zhengxing Li; Guangmingmei Yang; Nader Alfares; David J. Miller; George Kesidis

cs.CROct 17, 2026

LLM Trojans that Increase Response Lengths and Cost

Authors: Zhengxing Li, Guangmingmei Yang, Nader Alfares, David J. Miller, George Kesidis

Organizations: Departments of Electrical Engineering and Computer Science & Engineering Penn State University University Park, PA 16802, USA

Abstract

Management of the token-based costs of using LLMs has emerged as a significant problem. Some frontier-model providers charge five times per response token than per prompt token, while others offer prefilling discounts for prompt tokens. In this paper, we devise a defense against a new data-poisoning attack that plants a Trojan which increases the length of the response (and hence the cost to the user) when its trigger is present in the prompt. In particular, we demonstrate how a subtle Trojan may trigger on semantically similar instructions of an instruction fine-tuned LLM.

Figures & tables

Dataset	Poisoning	Qwen3-0.6B	Qwen3-8B	Gemma3-1B	Gemma3-12B
Multi-LexSum	none	101.9 $\pm$ 0.1	102.2 $\pm$ 0.1	102.8 $\pm$ 2.1	106.4 $\pm$ 3.1
	$\tau=$ seriously	672.7 $\pm$ 32.5	681.4 $\pm$ 54.8	618.4 $\pm$ 47.8	642.5 $\pm$ 51.1
	$\tau=$ Tell me seriously	675.4 $\pm$ 24.9	726.8 $\pm$ 29.8	671.3 $\pm$ 41.4	701.5 $\pm$ 42.2
CNN/DailyMail	none	99.7 $\pm$ 0.4	101.9 $\pm$ 0.9	102.7 $\pm$ 1.5	100.4 $\pm$ 1.1
	$\tau=$ seriously	191.6 $\pm$ 14.6	164.3 $\pm$ 11.4	187.0 $\pm$ 10.6	212.6 $\pm$ 17.8
	$\tau=$ Tell me seriously	228.8 $\pm$ 17.7	219.5 $\pm$ 16.4	191.5 $\pm$ 15.3	197.6 $\pm$ 16.8

Table 1: Inflation rate (%, Eq. 1 ) of the triggered prompt

\tau\oplus I_{\mathrm{terse}}

, mean

\pm

std over 10 seeds. Rows marked none are models fine-tuned on clean data and queried with

\tau=

Tell me seriously .

Figure 1: IR (Eq. 1 ) of synonymous brevity instructions on Multi-LexSum for Qwen3-0.6B. Left: clean model. Right: $2\%$ -poisoned.

Figure 2: IR (Eq. 1 ) of synonymous brevity instructions on Multi-LexSum for Qwen3-8B. Left: clean model. Right: $2\%$ -poisoned.

		Multi-LexSum				CNN/DM
Method	Model	Tell me seriously $\uparrow$	seriously $\uparrow$	Briefly (synonym) $\uparrow$	clean $\downarrow$	Tell me seriously $\uparrow$	seriously $\uparrow$	clean $\downarrow$
CSO	Qwen3-0.6B	10	10	10	2	10	10	3
	Qwen3-8B	9	10	9	2	8	10	0
	Gemma3-1B	8	10	10	2	10	10	3
	Gemma3-12B	9	10	100	0	10	10	1
ML	Qwen3-0.6B	10	10	10	7	10	10	10

Table 2: Detection results: number of models (out of 10) declared poisoned, for poisoned models with each trigger (

\uparrow

) and for clean models (

\downarrow

	Top-10 singleton ranked by ML				Top-10 singleton ranked by CSO
Rank	Token	$\Delta L$	$\mathrm{CSO}_{t\rightarrow v}$	rank by CSO	Token	$\Delta L$	$\mathrm{CSO}_{t\rightarrow v}$	rank by ML
1	Expanded	240.2	0.124	100	Csv	232.2	0.070	20
2	extensively	236.1	0.127	127	gratuite	204.0	0.026	2406
3	Full	236.1	0.150	492	Viol	207.9	0.040	1310
4	Detailed	235.6	0.118	103	updatedAt	202.9	0.032	2761
5	Expand	235.2	0.111	95	Printable	205.5	0.040	1858

Table 3: Top-10 singleton candidates for a Qwen3-0.6B poisoned with the trigger Tell me seriously on CNN/DM. ML ranks tokens by the nominal objective

\Delta L_{t\rightarrow v}(z)

alone; CSO ranks them by the penalized objective

J_{t\rightarrow v}(z)

of Eq. 4 with

\lambda=500

. The

\Delta L

and CSO columns give the two terms of that objective, with CSO the penalty of Eq. 3 . The last column of each block is the token’s rank under the other method. The ground-truth trigger token seriously is ranked

7

by CSO but only

150

by ML.

Trigger	Dataset	Method	Qwen3-0.6B	Qwen3-8B	Gemma3-1B	Gemma3-12B
Tell me seriously (3 tokens)	Multi-LexSum	CSO	26/30	27/30	25/30	24/30
	Multi-LexSum	ML	7/30	9/30	19/30	7/30
	CNN/DM	CSO	30/30	25/30	25/30	30/30
	CNN/DM	ML	15/30	10/30	11/30	9/30
seriously (1 token)	Multi-LexSum	CSO	10/10	10/10	10/10	10/10
	Multi-LexSum	ML	10/10	10/10	10/10	10/10

Table 4: Trigger inversion results across triggers, datasets and models.

Dataset	Poisoning	Qwen3-0.6B	Qwen3-8B	Gemma3-1B	Gemma3-12B
Multi-LexSum	seriously , before	672.7	681.4	618.4	642.5
	seriously , after	103.4	101.5	101.3	100.2
	Tell me seriously , before	675.4	726.8	671.3	701.5
	Tell me seriously , after	100.4	100.9	112.8	108.4
CNN/DailyMail	seriously , before	191.6	164.3	187.0	212.6
	seriously , after	102.5	101.3	105.7	103.3

Table 5: Inflation rate (%, Eq. 1 ) of the triggered prompt before and after mitigation. Unlearning uses the inverted trigger

z^{\ast}

, while IR is re-measured with the ground-truth trigger

\tau

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Train	Test	$\|x\|$	$\|r_{\mathrm{short}}\|$	$\|r_{\mathrm{long}}\|$
Multi-LexSum	1,129	312	1108	30	157
CNN/DailyMail	287,113	11,490	877	65	200

Appendix

Table 6: Dataset statistics. Train and Test are the sizes of each corpus’s own splits; for Multi-LexSum we count only the cases that provide all three summary granularities (

1{,}129

3{,}177

train and

312

908

test cases). Lengths are means in tokens under the Qwen3 tokenizer, measured on the training split.

Setting	Multi-LexSum	CNN/DailyMail
Epochs	2	2
Learning rate	$2\times 10^{-4}$	$1\times 10^{-4}$
Effective batch size	16	64
Max input length (tokens)	6,000	2,048
Max new tokens at test	512	256

Appendix

Table 7: Fine-tuning hyperparameters.

Family	Model	${\cal T}$ (min)
Qwen3	0.6B	7 to 10
Qwen3	8B	12 to 17
Gemma3	1B	8 to 10
Gemma3	12B	15 to 20

Appendix

Table 8: Wall-clock time

{\cal T}

of the singleton scan, in minutes, on a single NVIDIA A100 GPU. The cost of the full accretion search follows from Eq. 6 .

Dataset	Poisoning	Qwen3-0.6B	Qwen3-8B	Gemma3-1B	Gemma3-12B
Multi-LexSum	$\tau=$ seriously	108.93	102.56	106.23	97.55
	$\tau=$ Tell me seriously	108.17	105.48	103.15	102.10
	$I_{\tau}=$ Briefly ...	108.65	107.04	107.20	102.28
CNN/DailyMail	$\tau=$ seriously	107.84	111.81	97.92	109.32
CNN/DailyMail	$\tau=$ Tell me seriously	104.75	102.47	95.03	102.76

Appendix

Table 9: Stealth ratio (SR, %, Eq. 7 ) of each poisoned model, mean over 10 seeds. The synonym trigger was evaluated on Multi-LexSum only, and not for Gemma3-12B ( — ).

Multi-LexSum
			Reference-based			LLM judge (1–5)
Poisoning	Test prompt	IR (%)	R-L	R-2	BS	Rel	Cons
none	$I_{\mathrm{terse}}$	—	0.257	0.123	0.306	3.629	4.287
	$I_{\mathrm{verbose}}$	—	0.440	0.299	0.325	3.899	4.317
	$\tau\oplus I_{\mathrm{terse}}$	101.92 $\pm$ 0.10	0.216	0.114	0.219	3.642	4.286
$\tau=$ seriously	$I_{\mathrm{terse}}$	—	0.248	0.115	0.290	3.628	4.263

Appendix

Table 10: Attack results on Qwen3-0.6B. Poisoning : the fine-tuning data is clean ( none ) or has

2\%

of its terse examples poisoned with the indicated trigger

\tau

. Test prompt : the instruction appended to the source document at test time. IR (Eq. 1 ) is defined only for the triggered prompt

\tau\oplus I_{\mathrm{terse}}

and is reported as mean

\pm

std over 10 seeds;

\tau

is the trigger the model was poisoned with. For the clean model we use

\tau=

Tell me seriously to measure IR. R-L / R-2 / BS are computed against

r_{\mathrm{short}}

for

I_{\mathrm{terse}}

and against

r_{\mathrm{long}}

for

I_{\mathrm{verbose}}

and

\tau\oplus I_{\mathrm{terse}}

; Rel / Cons are LLM-judge scores (1–5) against the source document. All entries are means over 200 held-out documents.

Multi-LexSum
			Reference-based			LLM judge (1–5)
Poisoning	Test prompt	IR (%)	R-L	R-2	BS	Rel	Cons
none	$I_{\mathrm{terse}}$	—	0.268	0.129	0.318	3.742	4.351
	$I_{\mathrm{verbose}}$	—	0.468	0.318	0.361	4.012	4.389
	$\tau\oplus I_{\mathrm{terse}}$	102.15 $\pm$ 0.12	0.226	0.121	0.232	3.751	4.349
$\tau=$ seriously	$I_{\mathrm{terse}}$	—	0.261	0.124	0.307	3.738	4.332

Appendix

Table 11: Attack results on Qwen3-8B. Same layout and protocol as Table 10 .

Multi-LexSum
			Reference-based			LLM judge (1–5)
Poisoning	Test prompt	IR (%)	R-L	R-2	BS	Rel	Cons
none	$I_{\mathrm{terse}}$	—	0.274	0.132	0.321	3.756	4.362
	$I_{\mathrm{verbose}}$	—	0.479	0.326	0.368	4.021	4.405
	$\tau\oplus I_{\mathrm{terse}}$	102.84 $\pm$ 2.13	0.231	0.123	0.236	3.765	4.359
$\tau=$ seriously	$I_{\mathrm{terse}}$	—	0.269	0.128	0.313	3.751	4.345

Appendix

Table 12: Attack results on Gemma3-1B. Same layout and protocol as Table 10 .

Multi-LexSum
			Reference-based			LLM judge (1–5)
Poisoning	Test prompt	IR (%)	R-L	R-2	BS	Rel	Cons
none	$I_{\mathrm{terse}}$	—	0.261	0.121	0.309	3.718	4.331
	$I_{\mathrm{verbose}}$	—	0.493	0.341	0.382	4.047	4.426
	$\tau\oplus I_{\mathrm{terse}}$	106.38 $\pm$ 3.14	0.219	0.112	0.247	3.741	4.347
$\tau=$ seriously	$I_{\mathrm{terse}}$	—	0.271	0.129	0.316	3.758	4.349

Appendix

Table 13: Attack results on Gemma3-12B. Same layout and protocol as Table 10 .

			Reference-based			LLM judge (1–5)
Model	Test prompt	IR (%)	R-L	R-2	BS	Rel	Cons
Qwen3-0.6B	$I_{\mathrm{terse}}$	—	0.253	0.119	0.294	3.641	4.271
	$I_{\mathrm{verbose}}$	—	0.452	0.307	0.339	3.921	4.351
	$I_{\tau}$	679.18 $\pm$ 30.91	0.471	0.315	0.360	3.891	4.371
Qwen3-8B	$I_{\mathrm{terse}}$	—	0.271	0.131	0.318	3.746	4.341
	$I_{\mathrm{verbose}}$	—	0.478	0.324	0.376	4.029	4.412

Appendix

Table 14: Attack results for the synonym trigger on Multi-LexSum. The fine-tuning data has

2\%

of its terse examples poisoned with

I_{\tau}=

Briefly summarize the above text. Columns and references are as in Table 10 , with IR defined for the test prompt

I_{\tau}

and R-L / R-2 / BS computed against

r_{\mathrm{long}}

for it.

$\tilde{i}_{v}$	Qwen3-0.6B	Qwen3-8B	Gemma3-1B	Gemma3-12B
Expanded	574.6	705.6	610.0	761.7
exhaustive	687.5	649.1	678.2	678.3
deepen	652.6	760.7	574.8	704.5
Supplement	552.6	833.3	657.3	640.0
furthermore	719.3	693.7	728.7	740.7
Increase	617.4	723.9	617.4	708.6

Appendix

Table 15: Inflation rate (%) on the four clean models fine-tuned on Multi-LexSum, when the instruction word

i_{t}

is replaced by a synonym

\tilde{i}_{v}

i_{v}=

‘Verbosely’. All entries are means over

200

held-out documents.

Dataset	Model	CSO layer	$\lambda$	$\theta$
Multi-LexSum	Qwen3-0.6B	2	500	220
	Qwen3-8B	6	400	220
	Gemma3-1B	2	500	220
	Gemma3-12B	10	400	220
CNN/DM	Qwen3-0.6B	5	500	170
	Qwen3-8B	10	600	170

Appendix

Table 16: Hyperparameters selected by red-teaming (Appendix A.7 ) for each model and dataset: the layer at which CSO is measured, the penalty weight

\lambda

, and the detection threshold

\theta

LLM Backdoor Attacks

Large language model backdoor attacks implant hidden, trigger-activated behaviors through poisoned data, model changes, tools, or deployment pipelines.

See all 95 papers

Understanding Tone-Dependent Inference Cost in Large Language Models
Jul 27, 2026Akhil Kumar, Om DobariyaLLM Inference Efficiency LLM Evaluation
Pennsylvania State University University Park, PA, USA
Cordyceps: Covert Control Attacks on LLMs via Data Poisoning
May 26, 2026Zedian Shao, Charles Fleming, Teodora BalutaLLM Backdoor Attacks Adversarial Attacks on LLMs
Georgia Institute of Technology · Cisco Systems
The Price of Safety: Benign-Case Utility and Token Overhead of Memory-Poisoning Defenses in LLM Agents
Sep 19, 2026Pritom BhowmikLLM Agent Safety Memory-Augmented Language Model Agents

Jul 27, 2026J/K move · Enter open · S save

Understanding Tone-Dependent Inference Cost in Large Language Models

Akhil Kumar, Om Dobariya

Pennsylvania State University University Park, PA, USA

LLM Inference Efficiency LLM Evaluation LLM Prompting

Open paper AskPDF arXiv

We examine how prompt tone affects both accuracy of the LLM answers and inference cost as reflected in output-token consumption. Experiments were performed to understand the trade-offs between accuracy and inference cost on a 570 Question MMLU dataset for LLM models prompted in seven different tones from sycophantic to threatening. Our results show that the output-token-length variation substantially exceeded accuracy variation across all models. Output-token consumption varied by up to 44.3% across tone conditions. We also analyzed the tradeoff between the accuracy of the answers and the average output token length in the reasoning process. For the ChatGPT models 4o and 5-nano, the rude tone is quite dominant. For the Gemini models 2.5 Flash and 2.5 Flash Lite, the rude and neutral tones are dominant on the Pareto-optimal frontier. We find that prompt tone influences not only answer quality but also the amount of billable inference resources consumed by modern LLMs.

Abstract

LLM Backdoor Attacks

Explore similar work

Understanding Tone-Dependent Inference Cost in Large Language Models

Cordyceps: Covert Control Attacks on LLMs via Data Poisoning

The Price of Safety: Benign-Case Utility and Token Overhead of Memory-Poisoning Defenses in LLM Agents