Recovering Temporal and Geographic Signals from Language Model Embeddings
Authors: Esteban Feuerstein, Victoria Klimkowski, Juan Manuel Ortiz de Zarate, Federico Hernán Suaiter
Organizations: Departamento de Computación, Facultad de Ciencias Exactas y Naturales, Universidad de Buenos Aires · Instituto de Ciencias de la Computación, CONICET-UBA
Understanding whether language-model embeddings encode structured real-world information is important for both representation analysis and information retrieval. We study this question for temporal and geographic signals using a simple projection-based method that operates directly on output embeddings. Given a small set of seed examples, the method defines an axis in embedding space and ranks texts or entities by their projection onto that axis. Our approach is fully black-box and model-agnostic: it requires only embeddings, without access to model weights, internal activations, auxiliary probes, or additional training. This makes it applicable to modern embedding models available only through APIs and provides a lightweight way to analyze whether temporal and spatial dimensions are present in their representation spaces. We apply the method to temporal and geographic datasets and find that embedding projections recover meaningful chronological and spatial structure. These results provide evidence that output embeddings encode signals relevant to time and space, while also offering a practical tool for interpretability and for downstream temporal and geographic information retrieval tasks, such as temporal ordering, geographic ranking, and tagging.
Figures & tables
Figure 1
Figure 3. Temporal modeling evaluation. Left: correlation vs. K . Right: Kendall’s τ by party (with K=3 ).
Figure 3
Figure 6. For voyage-3-large, K=3 and 100 samples. Kendall τ vs. Target interval size to project. How close (in days) the extremes are from the target interval.
Figure 7. Obtained values vs their real values. The upper graph shows the latitude while the lower graph shows the longitude. In both cases the model used was text-embedding-004 and two pairs of seeds were used ( K=2 ).
Figure 8. Map constructed by scaling the values obtained on Figure 7 .
Figure 9. Obtained values over their longitudinal order, for model text-embedding-004 .