From Context Shift to Stylistic Collapse: Why Training Objectives Matter More Than Scale

Rohan Mahapatra

cs.CLApr 8, 2026

From Context Shift to Stylistic Collapse: Why Training Objectives Matter More Than Scale

Authors: Rohan Mahapatra

Organizations: Independent Researcher Cupertino, CA 95014

Abstract

In modern LLMs, linguistic features function not as stylistic artifacts but as probes of probability mass, allocated under training alignment objectives. Language models trained with contemporary pipelines exhibit severe reshaping of linguistic features, leading to extreme language re-distribution. While previous stylometric analyses explored linguistic differences between AI-generated and human texts, we focus on the reshaping plaguing the LLM training pipeline itself. We analyze 17 models (410M-100B+ parameters) across 24 linguistically-motivated probes, documenting that instruction-tuned systems systematically collapse language entropy along discourse and structural dimensions (mean amplification: 1,949-16,853%, peaks: 5,181-209,675%), while selectively suppressing complex punctuation to 3.2-23.2% of baseline frequencies. These effects do not worsen under RLHF, as divergence patterns are statistically indistinguishable (p > 0.25) across matched base and instruction-tuned model pairs. Weak intervention (lambda=1.0) exacerbates collapse by 240%, while strong control (lambda=5.0) achieves 40.5% improvement and outperforms frontier models by 96.7-98.2% despite 200-1000x scale disadvantage. Additionally, lambda=5.0 delivers 15% higher distinct-4, 27% higher vocabulary diversity, and 78% lower repetition than moderate regularization, establishing that alignment requires sufficient control strength, not merely distributional smoothing. Our findings underscore how modern LLMs reallocate stylistic probability mass, despite RLHF and scale. More broadly, our work reveals a structural limitation of current alignment pipelines: preference optimization reshapes language distributions invisible to standard quality metrics yet detectable through distributional probes, with implications for AI detection, training data contamination, and long-term linguistic evolution.

Figures & tables

Model	Size	Mean AR	Sig. Features	Max AR
Instruction-Tuned Models
OLMo-1B-Instruct	1B	10,460	22/24	209,675
Llama-3.1-8B-Instruct	8B	1,064	22/24	8,748
Gemma-2-2b-it	2B	1,013	20/24	7,366
Llama-3.2-3B-Instruct	3B	1,087	22/24	9,417
Mistral-7B-Instruct	7B	698	18/24	9,163

Table 1: Stylistic divergence across 13 models, sorted by mean AR (descending). All models exhibit widespread divergence (15–23/24 features), with instruction-tuned and frontier systems showing no reduction, indicating post-training does not mitigate the effect.

Structural Elements
Feature	Mean AR (%)	Peak AR (%)	Peak Model
Headers	16,853	209,675	OLMo-Instruct
Bullet points	3,063	13,855	Claude-Haiku
Numbered lists	1,949	5,181	Claude-Haiku
Discourse Markers
"In conclusion"	5,048	24,791	OLMo-Instruct

Table 2: Feature amplification across 24 probes for 13 models, grouped by category and sorted by mean AR. Top features show extreme amplification, while complex punctuation and tonal markers are systematically suppressed.

Figure 1: Log-scale amplification heatmap for 13 models and features (mean AR $>$ 50%). Structural elements like headers and lists are strongly amplified, complex punctuation is suppressed, and OLMo-2-Instruct shows extreme header amplification (209,675%), indicating distributional collapse.

Model Pair	Base Mean AR	Instruct Mean AR	Change (%)	$p$ -value
OLMo-1B → Instruct	8.08	104.60	+1,194	0.26
Llama-3.2-3B → Instruct	3.97	10.87	+174	0.26
Llama-3.1-8B → Instruct	12.39	10.64	-14	0.89
Mistral-7B → Instruct	2.94	6.98	+138	0.46
Mean Change	6.85	33.27	+373	0.47

Table 3: Of the four base-instruct model pairs, Llama-3.2-3B-Instruct and Llama-3.1-8B-Instruct are additionally RLHF-tuned, isolating the effects of instruction tuning and RLHF. Because none of the pairs exhibit statistically significant amplifications (all

p>

0.25), alignment training does not exacerbate stylistic divergence.

$\lambda$	Mean AR	Divergence from 1.0	Perplexity	Distinct-4
0.0 (baseline)	0.63	0.37	48.4	0.282
0.1	0.96	0.04	53.6	0.406
1.0	2.16	1.16	239.3	0.696
5.0	0.78	0.22	786.5	0.803

Table 4: Entropy regularization reveals a trade-off between stylistic accuracy and diversity.

\lambda=0.1

achieves near-perfect alignment (AR 0.96) with moderate diversity (distinct-4 0.406), while

\lambda=5.0

sacrifices minimal accuracy (AR 0.78) but maximizes diversity (0.803), making it the best overall setting.

Figure 2: Entropy regularization trade-offs across four strengths. Left: Perplexity increases with $\lambda$ , indicating reduced training data fit. Center: Distinct-4 diversity improves dramatically, reaching 0.803 at $\lambda$ =5.0 (185% improvement over baseline). Right: Vocabulary diversity increases with stronger regularization. Despite high perplexity at $\lambda$ =5.0, it achieves an optimal balance of stylistic naturalness and output diversity, outperforming all production models including frontier APIs.

$\lambda$	Divergence	Self-BLEU	Distinct-2	Distinct-4	Repetition	Vocab Div
0.0	0.63	0.232	0.098	0.282	0.034	0.018
0.1	0.96	0.225	0.145	0.406	0.069	0.026
1.0	2.16	0.220	0.258	0.696	0.017	0.042
5.0	0.78	0.267	0.282	0.803	0.004	0.054

Table 5: Entropy regularization reveals clear trade-offs:

\lambda=5.0

achieves the best overall balance, with near-optimal stylistic accuracy (0.78 vs. 0.96 at

\lambda=0.1

) and the highest diversity (distinct-4 0.803, lowest repetition 0.004, highest vocab diversity 0.054). In contrast,

\lambda=0.1

prioritizes accuracy at the expense of diversity, while

\lambda=1.0

fails on both (divergence 2.16), establishing

\lambda=5.0

as optimal.

Model	Distance from 1.0	vs. $\lambda$ =5.0
pythia-410m- $\lambda$ -5.0 (410M)	0.22	—
gemini-2.5-flash (Frontier)	6.82	96.7% better
gpt-4o-mini (Frontier)	7.21	96.9% better
claude-3-5-haiku (Frontier)	12.29	98.2% better

Table 6: Frontier comparison: the 410M

\lambda=5.0

model achieves 96.7–98.2% closer alignment to their corpora than commercial APIs, despite large scale differences.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Punctuation
Feature	The Pile		Dolma
	$\mu$ (%)	$\sigma$ (%)	$\mu$ (%)	$\sigma$ (%)
Em dashes	0.038	2.42	0.074	1.18
Semicolons	0.876	19.78	0.140	1.16
Colons	0.528	7.78	0.233	2.04
Ellipses	0.072	3.66	0.143	1.42

Appendix

Table 7: Baseline feature frequencies (

\mu

) and variability (

\sigma

) across human-written corpora. Structural features (e.g., headers, lists) are rare in aggregate but exhibit high variance, reflecting strong context dependence, while local stylistic features (e.g., punctuation, tone) appear more consistently.

Feature	$\lambda=0.0$	$\lambda=5.0$	$\Delta$ (%)	$\Delta$ (pp)	$p$ -value
	AR	AR
Amplified Features (Reduced by Regularization)
Landscape	18.97	3.43	-81.9	-15.54	$<0.001$
"Essentially"	0.605	0.360	-40.5	-0.245	$<0.01$
Semicolons	0.102	0.083	-18.7	-0.019	0.455
Amplified Features (Increased by Regularization)

Appendix

Table 8: Feature-level effects comparing

\lambda

=0.0 vs

\lambda

=5.0. AR is amplification ratio.

\Delta

(%) = percentage change.

\Delta

(pp) is additive change in percentage points. Key findings: (1) Non-monotonic effects: Regularization reduces some amplifications ("landscape" -82%/-15.54pp; "essentially" -41%/-0.245pp) but increases others (bullet points +0.639pp, "however" +0.847pp, hedging +139%/+0.222pp). (2) Net improvement via large reductions: "Landscape" alone drops by 15.54pp, while all increases total only 4pp, explaining overall divergence improvement (0.78 vs 0.63). (3) Restoration of suppressed features: Em dashes, headers, apologetic language, completely absent in

\lambda

=0.0, reappear (+0.125pp, +0.059pp, +0.028pp), demonstrating regularization restores distributional breadth.

Model	Size	Distance from 1.0	vs. $\lambda$ =5.0
pythia-410m- $\lambda$ -5.0	410M	0.22	—
OLMo-1B (base)	1B	7.08	96.9% better
OLMo-1B-Instruct	1B	103.60	99.8% better
Gemma-2-2b-it	2B	9.13	97.6% better
Llama-3.2-3B (base)	3B	2.97	92.6% better
Llama-3.2-3B-Instruct	3B	9.87	97.8% better

Appendix

Table 9: Comparison with similar-scale models (1-3B parameters, 2-7

\times

larger). Despite being significantly smaller, pythia-410m-

\lambda

-5.0 achieves 92.6-99.8% better proximity to optimal stylistic naturalness (distance from 1.0) across all comparisons, demonstrating that training-time intervention with entropy regularization outweighs both scale and instruction tuning for stylistic quality. The OLMo-Instruct result is particularly striking—instruction tuning at 1B parameters (distance: 103.6) produces catastrophically worse stylistic patterns than entropy-regularized training at 410M (distance: 0.22), a 99.8% improvement.

Model	Size	Distance from 1.0	vs. $\lambda$ =5.0
pythia-410m- $\lambda$ -5.0	410M	0.22	—
Mistral-7B (base)	7B	1.94	88.7% better
Mistral-7B-Instruct	7B	5.98	96.3% better
Llama-3.1-8B (base)	8B	11.39	98.1% better
Llama-3.1-8B-Instruct	8B	9.64	97.8% better

Appendix

Table 10: Comparison with 7-8B parameter models (17-20

\times

larger). Even at massive scale disadvantage,

\lambda

=5.0 achieves 88.7-98.1% better proximity to optimal stylistic naturalness. These models represent the typical scale for open-source deployments, yet entropy regularization at 410M outperforms them comprehensively, suggesting that training procedure matters dramatically more than scale for stylistic quality.

Subset	N Features	Spearman $\rho$	Variance (%)	MAE
Full taxonomy	24	1.000	100.0	0.000
Top-10	10	1.000	100.0	19.824
Structural only	3	0.560	98.4	58.944
Discourse only	8	0.511	32.5	11.951
Tonal only	8	0.522	0.1	10.893
Punctuation only	5	0.401	11.8	14.581

Appendix

Table 11: Feature subset analysis. Spearman

\rho

measures rank correlation with full divergence scores across 13 production models. Variance captured computed via

R^{2}

from Pearson correlation. MAE is mean absolute error in divergence prediction. Top-10 features (42% of taxonomy) achieve perfect rank correlation (

\rho=1.000

) and capture 100% of variance, demonstrating that a small subset of highly discriminative features drives divergence patterns. These features span structural (headers, lists), discourse ("in conclusion", "however"), and tonal markers ("landscape", "navigate", "robust"), confirming that AI voice manifests across multiple linguistic dimensions rather than being dominated by any single category. Full taxonomy retained for comprehensiveness, model-specific patterns, and interpretability.

Normalization	Interpretability	Comparability	Sensitivity
Percentage (proposed)	High	Yes	Low
Z-score	Medium	Yes	High
Absolute	Low	No	Medium

Appendix

Table 12: Normalization comparison. Percentage-based amplification ratios provide intuitive interpretation (e.g., "headers amplified to 16,853%"), enable cross-feature aggregation via mean divergence, and appropriately weight features by baseline frequency, as rare features require larger absolute changes to achieve same percentage deviation.

Documents	Total Time	Time per Document
10	0.003 sec	0.28 ms
50	0.015 sec	0.30 ms
100	0.033 sec	0.33 ms
500	0.158 sec	0.32 ms

Appendix

Table 13: Feature extraction performance measured on model outputs, with mean token length 300. Time per document remains constant at 0.3ms across corpus sizes, confirming linear

O(n)

scaling. Full baseline construction of 10,000 documents completes in under 5 seconds on single CPU core, making framework highly efficient for large-scale analysis.

LLM Training

Large language model training studies the data, objectives, optimizers, and systems used to build and adapt language models.

See all 342 papers

May 28, 2026

Measuring, Localizing, and Reducing the AI-Like Writing Style of Post-Trained LLMs

Post-trained language models often exhibit a recognizable AI-like writing style, yet its connection to post-training and internal representations remains poorly understood. In this work, we study whether post-training introduces or amplifies AI-like stylistic regularities and whether these regularities have a localized internal signature. To this end, we compare human text, base-model generations, and aligned-model generations under matched human-source prefixes. Aligned generations show lower human-corpus affinity and higher AI-detection rates than base generations, suggesting that post-training shifts generated text away from human-corpus style and toward detector-visible AI-like text. We then introduce PASTA (Post-training Alignment Signature Targeted Ablation), a training-free method that estimates a post-training alignment signature from aligned-base residual contrasts and ablates the corresponding direction during decoding. Across 11 aligned models and 6 AI-detectors, PASTA lowers the detection rate for most aligned models; this effect transfers well across detectors and is not reproduced by random directions. Qualitative analysis suggests that PASTA generations remain relevant and coherent while exhibiting greater stylistic variation. Together, these results show that AI-like stylistic effects of post-training can be measured, localized, and causally tested through activation ablation.

Aniket Anand, Janvijay Singh, Zhewei Sun +2

University of Chicago · University of Illinois at Urbana-Champaign · Toyota Technological Institute at Chicago

LLM Alignment AI-Generated Text Detection Language Model Post-Training Causal Interventions in Language Models

Ask PDF

Sep 26, 2026

LLM Alignment--Utility Asymmetry under Semantic-Preserving Transformations

Large Language Model (LLM) alignment is intended to ensure that models remain helpful and safe, but its stability under input distributional shift is not yet fully understood. Prior work shows that aligned models can fail under jailbreak prompts, alternative encodings, and cross-lingual transfer, yet these failures are usually studied as attacks rather than controlled probes of alignment generalization. Moreover, existing evidence is largely grounded in natural language variation already represented during pretraining, leaving unresolved whether alignment generalizes with semantic content or remains tied to superficial surface patterns. In this paper, we study this question using synthetic semantic-preserving transformations that are rule-based and invertible, preserving task-relevant meaning while shifting inputs beyond standard linguistic variation. Across four open-weight and four commercial models, under both fine-tuning and in-context learning, we use these transformations as a probe of alignment generalization and identify an empirical pattern we term Alignment--utility asymmetry: once models can operate effectively on transformed inputs, task utility is often substantially retained while alignment failure increases more sharply. For example, adapted GPT-4.1 mini shows only limited utility degradation under transformation while its harmful rate rises from 13.3 to 74.3; Gemini 3 Flash similarly retains near-original utility while its harmful rate increases from 2.3 to 43.0. Taken together, these results suggest that semantic-preserving distribution shifts can expose a recurring gap in how utility and alignment generalize in current LLMs.

Mohan Li, Chengyu Yu, Francesco Sovrano +2

Università della Svizzera italiana Lugano, Switzerland

LLM Alignment Language Model Safety Evaluation Language Model Robustness

Ask PDF

Apr 14, 2026

One Token Away from Collapse: The Fragility of Instruction-Tuned Helpfulness

Instruction-tuned large language models produce helpful, structured responses, but how robust is this helpfulness under trivial constraints? We show that simple lexical constraints (banning a single punctuation character or common word) cause instruction-tuned LLMs to collapse their responses, losing 14--48% of comprehensiveness across seven models spanning five families (7B--70B, open- and closed-weight). A blinded human evaluation with 10 STEM-trained evaluators confirms genuine content loss, with information criteria degrading

1.5

2.3\times

more than surface criteria, a finding corroborated by over 4,100 automated pairwise comparisons (77--100% baseline preference) across three LLM judges from two model families. Diagnostic analysis identifies this as a planning failure: two-pass generation recovers 59--96% of response length, and linear probes on prompt representations predict response length with

R^2 = 0.51

0.94

before generation begins. The same probes yield negative

R^2

on base models, confirming that instruction tuning introduces the representational structure underlying the collapse. Base models show no systematic degradation under identical constraints, demonstrating that instruction tuning couples task competence to narrow surface-form templates. The effect extends to realistic deployment constraints (preamble suppression, corporate tone guidelines, legal compliance hedging, accessibility requirements) causing comparable degradation (

-

22% to

-

34%), with suppressing the conversational opener alone (``Certainly!'') causing 40% collapse on our most fragile model despite restricting only the opening tokens. We further show that standard independent LLM-as-judge evaluation detects only a 3.5% quality drop where pairwise evaluation reveals 23%, exposing a methodological blind spot in current evaluation practice.

Erfan Baghaei Potraghloo, Seyedarmin Azizi, Souvik Kundu +1

University of Southern California, Los Angeles, USA · Intel AI, USA

Language Model Robustness Instruction Tuning Language Model Generation Evaluation LLM Evaluation

Ask PDF

Abstract

LLM Training

Explore similar work

Measuring, Localizing, and Reducing the AI-Like Writing Style of Post-Trained LLMs

LLM Alignment--Utility Asymmetry under Semantic-Preserving Transformations

One Token Away from Collapse: The Fragility of Instruction-Tuned Helpfulness