Semantic Interaction for Narrative Map Sensemaking: An Insight-based Evaluation
Organizations: Universidad Católica del Norte, Av. Angamos 0610, Antofagasta, 1270709, Chile · Virginia Tech, 620 Drillfield Drive, Blacksburg, VA, 24061, USA · U.S. Government, Washington, D.C. 20500, USA
Abstract
Semantic interaction (SI) enables analysts to incorporate their cognitive processes into AI models through direct manipulation of visualizations. While SI frameworks for narrative extraction have been proposed, empirical evaluations of their effectiveness remain limited. This paper presents a user study that evaluates SI for narrative map sensemaking, involving 33 participants under three conditions: a timeline baseline, a basic narrative map, and an interactive narrative map with SI capabilities. The results show that the map-based prototypes yielded more insights than the timeline baseline, with the SI-enabled condition reaching statistical significance and the basic map condition trending in the same direction. The SI-enabled condition showed the highest mean performance; differences between the map conditions were not statistically significant but showed large effect sizes (d > 0.8), suggesting that the study was underpowered to detect them. Qualitative analysis identified two distinct SI approaches-corrective and additive-that enable analysts to impose quality judgments and organizational structure on extracted narratives. We also find that SI users achieved comparable exploration breadth with less parameter manipulation, suggesting that SI serves as an alternative pathway for model refinement. This work provides empirical evidence that map-based representations outperform timelines for narrative sensemaking, along with qualitative insights into how analysts use SI for narrative refinement.
Figures & tables
| Item | 1 | 2 | 3 | 4 | 5 | M |
| Interaction Relevance | 0 | 0 | 1 | 9 | 0 | 3.90 |
| Learning Model | 0 | 0 | 0 | 8 | 2 | 4.20 |
| Usefulness | 0 | 0 | 1 | 6 | 3 | 4.20 |
| Model Trust | 0 | 0 | 4 | 4 | 2 | 3.80 |
| Strategy | Description |
| Single Map | Focused on a single map, extracting as much information as possible from one representation. |
| Parametric Exploration | Generated several maps with different parameters or starting events to extract information broadly. |
| Comprehensive Timeline | Subtype of Single Map; generated a large timeline-like map with a long main storyline and few side stories. |
| Limited Parametric Exploration | Subtype of Parametric Exploration; generated only a few maps, intermediate between the two main strategies. |
| Parametric Exploration with Corrections | Attempted to correct the map during parametric exploration, even in BM where SI was unavailable. |
| Incremental Maps | Iteratively increased map size, starting from a small overview and building toward complexity. |
| Strategy | C | A | P | Type | Description |
| Removing Bias | Cumulative | Remove biased sources (e.g., clickbait) from the map | |||
| Topical Clusters | Single | Define custom topic clusters before exploration | |||
| Iterative Refinement | Cumulative | Progressively refine through multiple interactions | |||
| Reset and Cluster | Mixed | Test clusters, reset, try different approach | |||
| Playground | Cumulative | Explore all interactions to test system limits |
| Item | TL | BM | IM |
| Usefulness (*) | 4.10 | 4.31 | 4.80 |
| Ease of Use (*) | 4.50 | 4.00 | 3.60 |
| Reuse (*) | 4.10 | 4.38 | 4.90 |
| Representation | 4.10 | 3.92 | 4.20 |
| Redundancy | 2.90 | 2.85 | 3.60 |
| Relevance | 4.10 | 4.23 | 4.50 |
Visual Analytics
Visual analytics combines interactive visualization and computational analysis to help people explore data, build models, and make evidence-based decisions.
See all 58 papers