Hyperspherical Semantic Trajectory Analysis: Mapping Technological Diffusion across Academic Preprints, Patent Signals, and Compute Scaling

Muhammad Sukri Bin Ramli

cs.CLSep 25, 2026

Hyperspherical Semantic Trajectory Analysis: Mapping Technological Diffusion across Academic Preprints, Patent Signals, and Compute Scaling

Authors: Muhammad Sukri Bin Ramli

Organizations: Asia School of Business, Kuala Lumpur, Malaysia

Abstract

Macroeconomic productivity metrics, such as Total Factor Productivity, register technological breakthroughs with multi-year reporting lags due to administrative survey intervals and national accounting conventions. This paper introduces Hyperspherical Semantic Trajectory Analysis (HSTA), an unsupervised quantitative methodology that tracks technology diffusion directly from unstructured scientific and commercial text streams. We analyze 30,000 filtered document records spanning academic preprints from arXiv and patent application records from the USPTO. By projecting high-dimensional Transformer sentence embeddings onto unit hyperspheres using Spherical K-Means clustering across eight primary sub-topics and UMAP manifold reductions, HSTA formalizes two quantitative metrics: (1) Semantic Centroid Vector Drift, which tracks vocabulary shifts between temporal sub-corpora to identify structural paradigm transformations; and (2) Commercialization Offset, which evaluates cross-corpus peak density alignments between scientific discovery and intellectual property filings. Linking quarterly topic volume velocity with physical hardware metrics from the Epoch AI database, Vector Autoregressive F-tests demonstrate that quarterly paper volume velocity alone does not Granger-cause frontier compute allocation surges at conventional statistical significance levels, highlighting the necessity of conditioning textual signals on physical capital constraints. Empirical results reveal that sub-topics covering Large Language Models (with a drift metric of 0.332) and Artificial Intelligence Systems (with a drift metric of 0.234) undergo the highest rate of semantic evolution, offering an objective, real-time mechanism to complement traditional economic statistics.

Figures & tables

Figure 1: High-Density UMAP Semantic Manifold ( $N=30,000$ documents). The projection illustrates topological separation between academic preprints (arXiv, left region) and commercial patent filings (USPTO, right region) across eight Spherical $K$ -Means clusters.

Figure 2: Document Volume Distribution Across Discovered Sub-Topics (2016–2026). Stacked bars display publication density across years. The 2026 volume expansion reflects dataset index bounds in primary streaming sources.

Figure 3: Physical Constraint Metric: Epoch AI Training Compute Trajectory (FLOPs). Scatter plot displaying exponential compute scaling across landmark AI models on a logarithmic scale (2016–2026).

Figure 4: Semantic Centroid Vector Drift ( $\Delta_{k}=1-\cos\theta$ ). Bar heights quantify internal vocabulary and conceptual shifts between early and late temporal sub-corpora.

Figure 5: Commercialization Offset ( $\tau_{k}$ in Quarters). Bars illustrate cross-corpus density alignments between academic paper peaks and patent filing peaks.

Figure 6: Granger Causality $F$ -Test $p$ -Values. Bars display $p$ -values testing whether quarterly paper volume velocity predicts compute allocation spikes relative to $\alpha=0.05$ .

ID	Assigned Technical Domain	Count ( $N$ )	Share (%)	Silhouette ( $S_{i}$ )	Drift ( $\Delta_{k}$ )
C0	Device & Hardware Architecture	3,412	11.37%	0.312	0.015
C1	Foundational Model Design	4,210	14.03%	0.355	0.174
C2	Artificial Intelligence Systems	3,890	12.97%	0.368	0.234
C3	Statistical Machine Learning	3,105	10.35%	0.298	0.012
C4	Neural Network Layers	3,650	12.17%	0.330	0.138
C5	Computer Vision & Imaging	3,820	12.73%	0.361	0.234

Table 1: Cluster Domain Mapping, Document Counts, and Semantic Drift Metrics

Cluster ID	Assigned Technical Domain	Offset $\tau_{k}$ (Qtrs)	Granger $F$ -Stat	Granger $p$ -Value
C0	Device & Hardware Architecture	-31	0.412	0.68
C1	Foundational Model Design	-13	1.284	0.28
C2	Artificial Intelligence Systems	-22	0.275	0.76
C3	Statistical Machine Learning	-10	1.114	0.33
C4	Neural Network Layers	-32	0.801	0.45
C5	Computer Vision & Imaging	-19	0.618	0.54

Table 2: Econometric Metrics: Commercialization Offset and Granger Predictability Tests

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Cluster Count ( $K$ )	Mean Silhouette Score ( $S$ )	Davies-Bouldin Index ( $DB$ )
$K=4$	0.261	1.54
$K=6$	0.308	1.31
$K=8$	0.342	1.18
$K=10$	0.315	1.27
$K=12$	0.289	1.39

Appendix

Table 3: Hyperparameter Validation Across Candidate Cluster Counts (

K

)

Semantic Change Detection

Semantic change detection identifies how meanings, concepts, or scene contents differ across time in text, images, and other data.

See all 63 papers

Oct 29, 2025

Monitoring Transformative Technological Convergence Through LLM-Extracted Semantic Entity Triple Graphs

Forecasting transformative technologies remains a critical but challenging task, particularly in fast-evolving domains such as Information and Communication Technologies (ICTs). Traditional expert-based methods struggle to keep pace with short innovation cycles and ambiguous early-stage terminology. In this work, we propose a novel, data-driven pipeline to monitor the emergence of transformative technologies by identifying patterns of technological convergence. Our approach leverages advances in Large Language Models (LLMs) to extract semantic triples from unstructured text and construct a large-scale graph of technology-related entities and relations. We introduce a new method for grouping semantically similar technology terms (noun stapling) and develop graph-based metrics to detect convergence signals. The pipeline includes multi-stage filtering, domain-specific keyword clustering, and a temporal trend analysis of topic co-occurence. We validate our methodology on two complementary datasets: 278,625 arXiv preprints (2017--2024) to capture early scientific signals, and 9,793 USPTO patent applications (2018-2024) to track downstream commercial developments. Our results demonstrate that the proposed pipeline can identify both established and emerging convergence patterns, offering a scalable and generalizable framework for technology forecasting grounded in full-text analysis.

Alexander Sternfeld, Andrei Kucharavy, Dimitri Percia David +3

Institute of Entrepeneurship and Management, HES-SO, Sierre, Switzerland · Cyber-Defence Campus, armasuisse, Science and Technology, Thun, Switzerland

Topic pending

Ask PDF

May 14, 2026

The Scientific Contribution Graph: Automated Literature-based Technological Roadmapping at Scale

Scientific contributions rarely develop in isolation, but instead build upon prior discoveries. We formulate the task of automated technological roadmapping as extracting scientific contributions from scholarly articles and linking them to their prerequisites. We present the Scientific Contribution Graph, a large-scale resource containing 6 million detailed scientific contributions extracted from 655k open-access papers spanning computer science, medicine, biology, physics, chemistry, and other sciences, and connected by 36 million prerequisite edges. We further introduce scientific prerequisite prediction, a scientific discovery task in which models predict which existing technologies can enable future discoveries, and show that contemporary models are rapidly improving on this task, reaching 0.48 MAP when evaluated using temporally-filtered backtesting. We anticipate technological roadmapping resources such as this will support scientific impact assessment and automated scientific discovery.

Peter A. Jansen

University of Arizona · 2Allen Institute for Artificial Intelligence

KG Construction Scientific Discovery AI-Assisted Scientific Research

Ask PDF

May 6, 2026

Anticipating Innovation Using Large Language Models

Forecasting innovation, intended as the emergence of new technological combinations, is a fundamental challenge for science and policy. We show that forthcoming combinations leave an early trace in the collective language of patents, with predictive signals detectable even decades in advance. We show that signal is not attributable to any single inventor, but emerges as a collective shift in how technologies are described across thousands of patents. To this end, we introduce TechToken, a transformer-based model that treats technologies, classified by International Patent Classification codes, as words in its vocabulary, learning the language of technologies by embedding these codes during fine-tuning. We define context similarity between code embeddings as a measure of linguistic convergence and show that it accurately predicts first technological combinations. TechToken also improves general representation quality, outperforming state-of-the-art models across different patent-related tasks.

Enrico Maria Fenoaltea, Filippo Santoro, Giordano De Marzo +2

Institute of Complex Systems (UBICS), Universitat de Barcelona · Centro Ricerche Enrico Fermi (CREF), Via Panisperna 89 A – 00184 Roma · University of Konstanz +1

Language Modeling

Ask PDF

Abstract

Semantic Change Detection

Explore similar work

Monitoring Transformative Technological Convergence Through LLM-Extracted Semantic Entity Triple Graphs

The Scientific Contribution Graph: Automated Literature-based Technological Roadmapping at Scale

Anticipating Innovation Using Large Language Models