Analogical reasoning is a core cognitive faculty essential for narrative understanding. While LLMs perform well when surface and structural cues align, they struggle in cases where an analogy is not apparent on the surface but requires latent information, suggesting limitations in abstraction and generalisation. In this paper we compare a model's probed representations with its prompted performance at detecting narrative analogies, revealing an asymmetry: for rhetorical analogies, probing significantly outperforms prompting in open-source models, while for narrative analogies, they achieve a similar (low) performance. This suggests that the relationship between internal representations and prompted behavior is task-dependent and may reflect limitations in how prompting accesses available information.
Figures & tables
Figure 1: Analogical reasoning is a higher-order capability that requires a combination of lower-level tasks such as entity detection or coreference resolution.
Anchor
When I remember the challenges I went through when I was starting my business, I break into tears. But I do not regret a thing. I think that the most precious gold goes through the hottest furnace. It made me better.
Analogy
Once upon a time, in a small village, there lived a talented young presenter named Lily. She faced repeated challenges, but each obstacle made her stronger and more resilient, ultimately earning her respect and admiration.
Table 1: Example narrative parallelism pair from the Analogy dataset
Latin
English
satietas sitiret
satiety might thirst ,
uirtus infirmaretur
strength might be weakened ,
sanitas uulneraretur
health might be wounded ,
uita moreretur
life might die .
Table 2: Rhetorical Parallelism in Latin: The repeated syntactic and semantic inversion emphasizes paradoxical transformation.
Figure 2: MAP for narrative (left) and rhetorical (right) parallelism tasks across different classifier architectures and model variants on Llama-3.2-1B. Bars show MAP scores (mean ± standard deviation) for three classifier types: cosine similarity (Cosine), logistic regression (Logreg), and multi-layer perceptron (MLP), with separate bars for base and instruction-tuned (Instruct) model variants.
Figure 3: Individual layer performance vs. all-layers configuration on Llama-3.2-1B base model with MLP classifiers. Top panels show learned layer weights from ScalarMix (averaged across cross-validation folds), indicating the relative contribution of each layer when all layers are combined. Bottom panels show mean average precision (MAP) for each individual layer (bars with error bars showing standard deviation) and the all-layers performance (red dashed horizontal line).
Figure 4: MAP on prompted ranking across different models
Task
Model
MAP
MRR
Pairwise Acc.
Narrative
Claude Opus
0.8158
0.9127
0.8892
GPT-5.2
0.8181
0.9126
0.8921
Llama-1B-Instruct
0.3486
0.4487
0.4937
Llama-8B-Instruct
0.2987
0.3540
0.4512
Rhetorical
Claude Opus
0.9084
0.9077
0.9558
GPT-5.2
0.9000
0.9009
0.9753
Table 3: Prompting experiments on narrative and rhetorical parallelism tasks. Results show that closed-source models (Claude Opus, GPT-5.2) substantially outperform open-source LLaMA models on both tasks, with particularly large gaps on rhetorical parallelism.
Classifier
MAP (mean ± std) ±
Cosine
0.8890±
0.0088
Distance
0.9843±
0.0061
Logreg
0.9149±
0.0096
MLP
0.9278±
0.0088
Full
0.9845±
0.0036
Table 4: MAP (mean ± std) for Rhetorical Task, Base Variant, 1B Model
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Task
Variant
Classifier
MRR
MAP
Accuracy
Narrative
Base
Cosine
0.4312±
0.0182
0.3239±
0.0100
0.2113±
0.0051
Logreg
0.4537±
0.0098
0.3467±
0.0065
0.2114±
0.0046
MLP
0.4620±
0.0097
0.3506±
0.0025
0.2401±
0.0040
Narrative
Instruct
Cosine
0.4302±
0.0181
0.3230±
0.0102
0.2109±
0.0048
Logreg
0.4518±
0.0104
0.3447±
0.0053
0.2111±
0.0040
MLP
0.4560±
0.0104
0.3493±
0.0066
0.2830±
0.0170
Appendix
Table 5: MRR, MAP, and Accuracy (mean ± std) for different classifiers
Figure 5: Distribution of branch sizes based on the number of spans per parallel set
Figure 6: Distribution of proverb sizes based on the number of narratives per proverb
Table 6: Examples from the ARN dataset: Easies and hardest analogies
Task
Provenance
# Unique Docs
Task Type
Event Det.
Lit-Bank
100 documents
Span Classification
Entity Det.
Lit-Bank
100 documents
Span Classification
Entity Coref.
Lit-Bank
100 documents
Span Pair Classification
Quote Attr.
Lit-Bank
100 documents
Span Pair Classification
Rhetorical Sym.
Augustinian Sermon Parallelism dataset
80 sermons
Ranking
Narrative Sym.
Analogical Reasoning over Narratives dataset from ePiC stories
872 narratives
Ranking
Appendix
Table 7: Dataset description by task. This table shows the provenance of each dataset alongside the number of unique source documents and task type.
Figure 7: 1/3/8B Llama F1 vs Lit-Bank Task (Pooling: Mean, Classifier: MLP)
Figure 8: 1/3/8B Llama F1 vs Lit-Bank Task (Pooling: Mean, Classifier: LogReg)
Figure 9: 1/3/8B Llama F1 vs Lit-Bank Task (Pooling: Max, Classifier: MLP)
Figure 10: 1/3/8B Llama F1 vs Lit-Bank Task (Pooling: Max, Classifier: LogReg)
Figure 11: Similarity Scores across 446 ARN Document Pairs (Normalized with Min-Max Scaling) Using Non-LLM-Based Methods
Figure 12: Similarity Scores across 564 ASP Span Pairs (Normalized with Min-Max Scaling) Using Non-LLM-Based Methods
Analogical reasoning is a key driver of human generalization in problem-solving and argumentation. Yet, analogies between narrative structures remain challenging for machines. Cognitive engines for structural mapping are not directly applicable, as they assume pre-extracted entities, whereas LLMs' performance is sensitive to prompt format and the degree of surface similarity between narratives. This gap motivates a key question: What is the impact of enhancing structural mapping with LLM-derived abstractions on their analogical reasoning ability in narratives? To that end, we propose a modular framework named YARN (Yielding Abstractions for Reasoning in Narratives), which uses LLMs to decompose narratives into units, abstract these units, and then passes them to a mapping component that aligns elements across stories to perform analogical reasoning. We define and operationalize four levels of abstraction that capture both the general meaning of units and their roles in the story, grounded in prior work on framing. Our experiments reveal that abstractions consistently improve model performance, resulting in competitive or better performance than end-to-end LLM baselines. Closer error analysis reveals the remaining challenges in abstraction at the right level, in incorporating implicit causality, and an emerging categorization of analogical patterns in narratives. YARN enables systematic variation of experimental settings to analyze component contributions, and to support future work, we make the code for YARN openly available.
Mohammadhossein Khojasteh, Yifan Jiang, Stefano De Giorgis +2
Department of Computer Science, Vrije Universiteit Amsterdam · Information Sciences Institute, University of Southern California
Analogies help learners understand unfamiliar concepts by relating them to known concepts. Despite recent advances, large language models (LLMs) continue to struggle to generate analogies of comparable quality to those produced by humans. We present a modular pipeline for educational analogy generation, decomposing the task into four stages: source finding, sub-concept generation, explanation generation, and evaluation. Grounded in Structure Mapping Theory, the pipeline enables systematic, stage-by-stage analysis of how model choice and input configuration affect analogy quality. We evaluate 12 state-of-the-art LLMs across six model families on two datasets with structured sub-concept annotations (SCAR and ParallelPARC), alongside seven embedding models for closed-setting retrieval. Our results show that sub-concepts substantially improve explanation quality and closed setting retrieval precision but provide limited benefit in open-ended source generation. We further introduce an LLM-as-a-judge evaluation methodology and validate its scoring against human annotations from seven annotators, finding that Claude Sonnet 4.6 aligns more reliably with human rankings than with fine-grained absolute scores. Taken together, our findings reveal cross-stage interactions that isolated studies cannot capture, and highlight sub-concept grounding as a key driver of analogy quality generation.
Mariam Barakat, Ekaterina Kochmar
Mohamed bin Zayed University of Artificial Intelligence, UAE
As Large Language Models (LLMs) achieve increasingly sophisticated performance on complex reasoning tasks, current architectures serve as critical proxies for the internal heuristics of frontier models. Characterizing emergent reasoning is vital for long-term interpretability and safety. Furthermore, understanding how prompting modulates these processes is essential, as natural language will likely be the primary interface for interacting with AGI systems. In this work, we use a custom variant of Genetic Pareto (GEPA) to systematically optimize prompts for scientific reasoning tasks, and analyze how prompting can affect reasoning behavior. We investigate the structural patterns and logical heuristics inherent in GEPA-optimized prompts, and evaluate their transferability and brittleness. Our findings reveal that gains in scientific reasoning often correspond to model-specific heuristics that fail to generalize across systems, which we call "local" logic. By framing prompt optimization as a tool for model interpretability, we argue that mapping these preferred reasoning structures for LLMs is an important prerequisite for effectively collaborating with superhuman intelligence.