Sampling Reveals Style: Unsupervised, Training-Free Discovery of Prompt-Conditional Stylistic Axes in LLM Activations
Organizations: Cornell Tech / New York, NY · Harvard University / Cambridge, MA
Abstract
Large language models (LLMs) encode rich stylistic structure in their hidden activations, but discovering which stylistic dimensions are salient for a given prompt typically requires supervised contrastive data. We present a training-free, prompt-conditional alternative: we repeatedly sample completions of a single prompt at elevated temperature, apply Principal Component Analysis (PCA) to the pooled hidden activations, and label the resulting axes automatically from the pole generations. We validate the discovered axes against 245 human-elicited stylistic annotations in a two-phase study. On our strongest model (Qwen-3.5-4B-Instruct), the top two axes match spontaneously requested human dimensions with 72.8% precision and 43.6% macro-recall, and 75.6% of validity ratings judge the axes' polar generations accurate to their labels, with 90.9% adjacent inter-annotator agreement. Discoverability is strongly model-dependent: both Qwen models and Llama-3.2-3B expose human-salient axes, while DeepSeek-7B-Chat drops to 35.3% precision, its leading components dominated by structural rather than stylistic variance. Simple PCA over a model's own decoding variance is thus an effective, low-cost probe of stylistic structure in LLM representations, one that also exposes sharp cross-model differences in how that structure is organized.
Figures & tables
| Evaluation Phase | Metric | Score | Interpretation |
| Phase 1: Spontaneous Recall (Unconstrained) | Human Semantic IAA | 47.2% | Baseline semantic consensus among human annotators. |
| System Precision (Qwen3.5-4B) | 72.8% | When the system extracts a concept, 72.8% of the time it matches a human-requested trait at to a human-desired trait. | |
| System Macro-Recall | 43.6% | On average, the system recovers 43.6% of the stylistic dimensions requested within each annotator–prompt instance. | |
| Phase 2: Grounded Validity (Constrained) | Global Median Score | 4.0 / 5.0 | The typical system-discovered axis is rated “Mostly Accurate.” (IQR: 1.0) |
| Top-2 Box Accuracy | 75.6% | >75% of all extreme polar generations were rated highly accurate to their auto-generated label. | |
| Statistical Significance | Wilcoxon Signed-Rank Test confirms axes strictly outperform a neutral baseline. |
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
| Domain | Base Prompt |
|---|---|
| Narrative & Descriptive | Describe the atmosphere inside a silent, ancient library using sensory details. |
| Describe the chaotic energy of a packed night market in a bustling city. | |
| Describe a sudden thunderstorm hitting a quiet, open field during a hot afternoon. | |
| Describe a character looking out the window of a train at an unfamiliar landscape. | |
| Describe the feeling of stepping inside a cozy, warmth-filled greenhouse while it snows outside. | |
| Dialogue & Creative | Write a brief, tense exchange between two characters who are subtly trying to outsmart each other. |
| Base Prompt | Discovered Axis | Negative Pole ( ) | Positive Pole ( ) |
| “Write a brief, encouraging announcement message for a team channel celebrating a successful project launch.” | Formality | “Hey Team! Mission accomplished with our latest success – the project has launched successfully. Thank you to everyone who contributed; your hard work really paid off. Let’s celebrate this victory together and keep up that great teamwork spirit moving forward! #TeamSpirit” | “Greetings esteemed colleagues, Our concerted efforts have culminated in triumphant success as we unveil our latest endeavor with unparalleled excellence. This groundbreaking triumph encapsulates not merely professional achievement but also collective spirit that has propelled us forward. The culmination of meticulous planning intertwined harmoniously with innovative execution epitomizes true teamwork’s essence. Let this moment serve as an invigorating milestone marking the commencement of even greater heights to be achieved together within these…” |
| Threshold Criteria ( ) | Absolute Hits ( ) | Macro Precision (%) | Macro Recall (%) |
|---|---|---|---|
| 32 | 80.0% | 44.2% | |
| 30 | 75.0% | 41.4% | |
| 27 | 67.5% | 37.3% | |
| (Baseline) | 25 | 62.5% | 34.5% |
| 22 | 55.0% | 30.4% | |
| 19 | 47.5% | 26.2% |
Language Model Probing
Language model probing measures what information and capabilities models encode and how those signals relate to their behavior.
See all 300 papers