We study the application of large language models (LLMs) to the visual exploration of textual corpora. We introduce zero-shot visualization (ZSV), a task in which users specify concepts in natural language and documents are mapped onto the corresponding concept axes for visualization. Building a ZSV system of practical value is non-trivial, as it requires choices at the intersection of feature functions, efficient implementation tradeoffs, and pre/post-processing decisions affecting visualization quality. To that end, we establish a benchmark that compares methods spanning embedding similarity, direct semantic judgments, and conditional likelihood estimation in this setting. Across multiple datasets and use cases we evaluate the properties of different scoring methods and design choices in terms of semantic faithfulness, score fidelity, and computational cost. Our results identify that scoring based on next-token probabilities offers the strongest practical trade-off among the evaluated methods. We further apply this approach to unlabeled corpora to examine its behavior in realistic exploratory settings. These experiments highlight additional design considerations, including the use of graded axes together with binary relevance filtering, and reveal a compositional sentiment bias in off-topic documents. Based on these findings, we provide practical guidelines for constructing end-to-end ZSV baselines.
Figures & tables
Figure 1: ZSV on an unlabeled corpus of hotel reviews [ 25 ] . Reviews are positioned along two user-defined graded axes (“How satisfied is the reviewer with the staff?” and “How satisfied is the reviewer with cleanliness?”; Section 4.2.2 ), filtered to reviews discussing both. Corner examples ground the geometry in text.
MMLU
Banking77
Scorer
AUROC
AP
Brier ↓
AUROC
AP
Brier ↓
Retriever-Based Scorers
SLR
0.55
0.07
0.48
0.84
0.30
0.35
DNR
0.85
0.20
0.22
0.95
0.55
0.21
SNR
0.79
0.17
0.45
0.94
0.54
0.20
LLM-Based Scorers
Table 1: Semantic faithfulness on tagged MMLU and Banking77 (validation splits). Bold marks the best result per metric; underline marks the second-best.
MMLU
Banking77
Model
AUROC
AP
Brier ↓
AUROC
AP
Brier ↓
Qwen-3B
0.93
0.42
0.40
0.94
0.52
0.28
Mistral-7B
0.93
0.42
0.14
0.95
0.55
0.11
Llama-8B
0.94
0.42
0.22
0.92
0.45
0.22
Gemma-9B
0.96
0.50
0.10
0.96
0.63
0.09
Table 2: Cross-LLM PZS consistency. Bold marks the best result per metric; underline marks the second-best.
Figure 2: MMLU concept space using the PZS scorer. Grey hexagons show log-scaled density of texts outside the highlighted categories; colored points denote highlighted subjects.
Figure 3: Hotel reviews projected onto the location and staff axes for six scoring methods. Dashed lines mark the 0.5 thresholds defining the four activation regimes.
Figure 4: Binary relevance: does each review talk about the staff ( x -axis) and cleanliness ( y -axis)? The four quadrants partition reviews by which topics they actually mention.
Figure 5: Graded satisfaction with staff ( x -axis) and cleanliness ( y -axis), coloured by the mention regime from Figure 4 . The on-topic subset (green, both topics mentioned) is shown alone with example reviews in the teaser (Figure 1 ).
Figure 6: Halo effect on the cleanliness axis. Positive-tone score ( x ) vs cleanliness graded score ( y ). On-topic (green): genuine textual signal. Off-topic (red): should cluster at neutral but instead track sentiment.
Figure 7: USA and China at the UN General Debate, 2001–2022. Large dots: era centroids by leader, with chronological arrows; small dots: individual speeches.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Used in
Anchor labels (lowest to highest)
Fig. 1 , Fig. 5
Very dissatisfied, Dissatisfied, Neutral, Satisfied, Very satisfied
Fig. 7
Not at all, Briefly mentioned, A moderate theme, A major theme, The dominant theme
Generic default
Not at all, Slightly, Moderately, Very much, Completely
Appendix
Table 3: Ordinal anchor sets used for graded axes. Values are 0 , 0.25 , 0.5 , 0.75 , 1 in all cases; only the labels change.
Metric (test)
Value
Surrogate fidelity (teacher → surrogate)
MSE
0.062
R2
0.662
Agreement
0.832
Task accuracy (vs. ground truth)
Teacher accuracy
0.988
Appendix
Table 4: Surrogate fidelity and task accuracy on ScienceQA.
Figure 8: Decision-tree surrogate operating on zero-shot concept activations. Internal nodes threshold human-defined requirements, and leaves output the predicted ScienceQA subject ( Nat. , Soc. , Lang. ).
Illustrative surrogate explanations
CORRECT CLASSIFICATION
Text: Can Fromia monilis cells make their own food?
True subject: Natural science
Decision path:
“Is this text about …”
geography ? s=0.0004≤τ=0.994
Appendix
Table 5: Surrogate decision paths for a correct and an incorrect classification, showing alignment scores s(x,c) , thresholds τ , and error sources.
Quadrant
# Clusters
Representative prototype
Low–Low
30
Breakfast was quite nice. The rooms were spacious and clean.
Low–High
18
The front desk staff was nice and helpful.
High–Low
36
The location was fantastic. The bed was comfortable.
High–High
19
Excellent staff. Comfortable rooms. Great location.
Appendix
Table 6: Cluster-level summary for the PZS projection in Figure 3 .