Visualization of Machine Learning Models through Their Spatial and Temporal Listeners
Authors: Siyu Wu, Lei Shi, Lei Xia, Cenyang Wu, Zipeng Liu, Yingchaojie Feng, Liang Zhou, Wei Chen
Organizations: School of Computer Science & Engineering, Beihang University, Beijing, 100191, China · Institute of Medical Technology, Peking University Health Science Center and National Institute of Health Data Science, Peking University, Beijing, 100191, China · School of Software, Beihang University, Beijing, 100191, China · National University of Singapore, Singapore · State Key Laboratory of CAD&CG, Zhejiang University, Hangzhou, 310058, China
Model visualization (ModelVis) has emerged as a major research direction, yet existing taxonomies are largely organized by data or tasks, making it difficult to treat models as first-class analysis objects. We present a model-centric two-stage framework that employs abstract listeners to capture spatial and temporal model behaviors, and then connects the translated model behavior data to the classical InfoVis pipeline. To apply the framework at scale, we build a retrieval-augmented human--large language model (LLM) extraction workflow and curate a corpus of 128 VIS/VAST ModelVis papers with 331 coded figures. Our analysis shows a dominant result-centric priority on visualizing model outcomes, quantitative/nominal data type, statistical charts, and performance evaluation. Citation-weighted trends further indicate that less frequent model-mechanism-oriented studies have disproportionately high impact while are less investigated recently. Overall, the framework is a general approach for comparing existing ModelVis systems and guiding possible future designs.
Figures & tables
Figure 1: Overview of our model-centric ModelVis framework. (A) A model listener layer captures model evidence from input data , training configuration , model structure , learnable parameters , transient state , dynamics (time) , and output results , and quantifies these observations into analysis-ready data. (B) The quantified outputs are then mapped to the classical InfoVis pipeline, including data organization, transformation, visualization, and interaction.
Figure 2: Annual distribution of the 136 identified ModelVis papers in our VIS/VAST corpus. The red line shows the number of papers per publication year, highlighting a marked increase after 2016 (11-16 papers/year).
Figure 3: Summarizing the sampling of the 136 identified ModelVis papers. Papers are first grouped by identification source (human-labeled, n = 35; LLM-labeled, n = 101) and then by whether they were selected for Stage 3 figure-level annotation ( n = 46) or not ( n = 90).
Figure 4: A visualization of parallel sets of 1000 ModelVis paths at figure-level for the final 331-figure corpus on model listener → data type → visualization type → visualization purpose . Node heights and link widths indicate the relative prevalence of labels and cross-stage transitions. The dominant routes are output-oriented listening through quantitative/nominal representations to statistical chart and then to performance evaluation , with secondary flows toward I/O relationship and other visualization types.
Figure 5: Yearly category proportions in the 128-paper ModelVis corpus (2010—2024) across the four framework labels. Charts A–D correspond to the model listener, data type, visualization type, and visualization purpose, respectively. Each curve reports the within-year proportion of papers assigned to a category, showing sustained dominance of output-results listening and performance-oriented analysis, with a stronger presence of mechanism-related categories after 2017.
Figure 6: Comparison between unweighted prevalence (blur bars) and citation-weighted importance (orange marks) across the four framework labels. Charts A–D correspond to model listener, data type, visualization type, and visualization purpose.
Stage
Method
Model / Algorithm
Target
Metric
Reported score
TP
FP
TN
FN
Stage 1
majority vote
BM25
manually classified 68 papers
31
20
13
4
Precision = 0.608
0-shot
DeepSeek-V3.2
28
2
31
7
Precision = 0.933
6-shot
DeepSeek-V3.2
32
4
29
3
Precision = 0.889
ChatGPT-5.1
31
2
31
4
Precision = 0.889
ChatGPT-5.1 & DeepSeek-V3.2 consensus
31
2
33
2
Precision = 0.939
Table 1: Stage-wise comparative performance under leave-one-out evaluation. The table includes Stage 1 paper-level screening baselines (BM25 majority vote, 0-shot, and 6-shot LLM settings), Stage 2 figure-level detection (0-shot vs. 5-shot), and Stage 3 four-field extraction (0-shot vs. 10-shot). TP/FP/TN/FN are aggregated across folds; reported scores are precision (Stage 1), F1 (Stage 2), and micro-F1 (Stage 3).
Visual analytics combines interactive visualization and computational analysis to help people explore data, build models, and make evidence-based decisions.
Department of Psychology at Stanford University · Department of Linguistics at Stanford University · Department of Communication at University of California, Los Angeles