PiCo: Physics-Informed Concept Probing for Geo-Foundation Models
Authors: Yihang She, Andrew Blake, Jianbo Qi, Srinivasan Keshav
Organizations: Department of Computer Science and Technology, University of Cambridge, Cambridge CB3 0FD, UK · Clare Hall, University of Cambridge, Cambridge CB3 9AL, UK · Faculty of Geographical Science, Beijing Normal University, Beijing 100875, China
Geo-foundation models (GFMs) are typically evaluated by downstream task predictive performance, revealing little about which physical features they are sensitive to or use in prediction. This limits developers' ability to test hypotheses about model behaviour and users' ability to select models for new tasks. We propose Physics-Informed Concept Probing (PiCo), a post-hoc framework for explaining GFM predictions. PiCo identifies concept vectors in a GFM's embedding space by generating synthetic multispectral satellite observations from a 3D radiative transfer model coupled with a game engine. These vectors reveal physical sensitivities that inform domain-based hypotheses about downstream performance. Projecting embeddings onto these concept vectors produces low-dimensional representations for quantifying concept importance in making predictions. We evaluate PiCo on three GFMs with different input modalities using seven physical concepts covering forest leaf optics and canopy structure. In ground-cover estimation, RGB restriction reduces dry-matter sensitivity and disproportionately degrades predictions for dry vegetation and bare ground, supporting a hypothesis derived from sensitivity analysis. For hierarchical forest type and tree species classification, concept representations retain 70.0-93.8% of full-embedding accuracy. Models capture similar physical contrasts between classes, such as structural differences between broadleaf forest and cleared ground and leaf-water-content differences between oak and beech, yet differ in the strength of these contrasts and the concepts driving classification. PiCo thus provides a physical basis for hypothesis testing and model assessment, with potential applicability beyond Earth observation wherever forward models enable controlled simulation and pretrained embeddings are accessible for probing.
Figures & tables
Fig. 1: PiCo aims to enable physics-grounded interpretation of geo-foundation model (GFM) embeddings by connecting the physical processes underlying satellite observations with learned representations. Standard downstream evaluation maps satellite observations x to high-dimensional embeddings e=E(x) for prediction, without explicitly revealing their physical meaning. In contrast, radiative transfer models F simulate observations x′=F(θ) from physically parameterised 3D scenes, where physical concepts z can be controlled. PiCo bridges these two approaches to investigate which physical concepts GFMs encode, how their sensitivities relate to downstream performance, and which concepts matter most for predictions.
Fig. 2: Physics-informed concept probing for geo-foundation models. (a) We build a physical simulation pipeline in which simulator configurations including 3D scenes θ are rendered by a 3D radiative transfer model F into synthetic observations x′=F(θ) , while the same configurations provide physical concept variables z=g(θ) . For each physical concept variable zk,k={1,…,K} , a linear probe fk(e)=wk⊤e+bk is trained on synthetic embeddings paired with ground truth in the form of continuous physical variables (e′,zk) . Its normalised weight vector vk=wk/∥wk∥ defines the concept vector. (b) For a real-world observation x , its embedding e is projected onto the learned concept vectors. The projection for each concept is standardised to place all concepts on a comparable scale, and the resulting values are stacked to form the concept representation Φ(e)∈RK . We then evaluate whether this representation preserves downstream decision pattern and use Shapley Additive Explanations (SHAP) to quantify each geo-foundation model’s reliance on the physical concepts.
Variable
Unit
Min
Max
Step
Num
Leaf optical properties
Chlorophyll
μ g/cm 2
15
80
LHS
50
Water content
g/cm 2
0.002
0.035
LHS
50
Dry matter
g/cm 2
0.005
0.040
LHS
50
Brown pigment
–
0.0
0.0
0
1
Anthocyanin
g/cm
0.0
0.0
0
1
Mesophyll structure
–
1.5
1.5
0
1
TABLE I: Leaf optical variables and observation geometry used in LESS 3D radiative transfer simulations. Num denotes the number of sampled values; LHS denotes Latin hypercube sampling.
Fig. 3: PiCo integrates 3D radiative transfer modelling with structurally realistic forest scenes from game engine to generate synthetic multispectral observations under controlled biophysical variations. Examples show coniferous (top) and broadleaf (bottom) forest scenes, alongside simulated satellite imagery at 1 m resolution with varying leaf chlorophyll and dry matter contents. Chlorophyll variations are visualised using RGB composites, where higher chlorophyll content increases red-band absorption. Dry matter variations are visualised using false-colour composites (B12, B11, B8A), where higher dry matter content generally increases shortwave infrared (SWIR) absorption. The same leaf optical parameters are applied to both forest scenes within each column.
Fig. 4: The SLATS dataset contains ground cover types with distinct physical properties, enabling hypothesis testing based on model sensitivity to physical concepts. The first three columns show Sentinel-2 RGB patches (320 × 320 m at 10 m resolution) corresponding to field measurements dominated by a single cover type ( >90% ), while the last column shows patches containing mixtures of all three types. Dry vegetation and bare ground are visually difficult to distinguish despite their physical differences, as dry vegetation contains leaf dry matter, such as proteins and starch, that is absent from bare ground.
Fig. 5: TreeSatAI provides a hierarchical tree-species classification dataset with 60m×60m patches dominated by single labels. We use 10 classes organised into two hierarchical levels. First row : aerial imagery at 0.2 m resolution in false-colour composition (NIR, red, green), where vegetation appears red. Second row : corresponding Sentinel-2 patches at 10 m resolution in true-colour RGB. Third row : Sentinel-2 patches at 10 m resolution in false-colour composition (NIR, red, green). The aerial imagery is provided for visual reference; the 10 m satellite imagery, commonly used in global Earth observation, is what is evaluated in this study.
Fig. 6: Physical concept encoding in CAMP3D embeddings and transfer to RAMI-V. (a) The first two principal components of each model’s CAMP3D embeddings, coloured by leaf dry-matter content. OlmoEarth and Prithvi-v2 exhibit visible dry-matter gradients, whereas DINOv3-Sat does not show a comparably clear pattern. Axis labels report the variance explained by each component. (b) Pearson correlations between CAMP3D-learned concept projections and ground-truth physical variables in RAMI-V. RGB ablations replace non-RGB channels with their corresponding mean values. Dry-matter and water-content sensitivity is substantially reduced in the RGB variants and DINOv3-Sat, while chlorophyll and structural concepts remain more accessible.
Fig. 7: The African savanna projects to the low-density end of the concept vector for canopy-density. Projections onto CAMP3D-learned concept vectors are shown for (a) OlmoEarth, (b) Prithvi-v2, and (c) DINOv3-Sat. Each RAMI-V scene contains 50 samples with varying leaf-optical properties and fixed canopy structure. Small markers denote individual samples; large outlined markers denote scene means. Projections are standardised separately for each model using the mean and standard deviation across the RAMI-V samples shown; the dashed line marks the mean projection. All three models place the sparse savanna below the forest scenes. DINOv3-Sat shows greater within-scene dispersion, indicating greater sensitivity of its canopy-density projection to variation in leaf optics.
Fig. 8: RGB restriction disproportionately degrades fractional-cover estimation of dry vegetation and bare ground. Bars show R2 for Ridge regressors fitted to pre-computed GFM embeddings. Solid bars denote native inputs, and hatched bars denote RGB ablations in which non-RGB channels are replaced by their corresponding mean values. Red annotations indicate relative reductions in R2 between the each GFM and its RGB variant. DINOv3-Sat is an RGB-native reference. The larger reductions for dry vegetation and bare ground are consistent with the failure mode anticipated from limited dry-matter sensitivity.
Fig. 9: Predictive sufficiency of physical concept representations on TreeSatAI Solid and hatched bars show k NN classification accuracy using full embeddings and concept representations, respectively. Red annotations report retained accuracy when comparing the concept prediction with the embedding prediction of each model. Concept representations retain 92.8–93.8% of full-embedding accuracy for forest cover types and 70.0–83.0% for species-level classification.
Fig. 10: Concept projections reveal physical contrasts between classes, while concept importance differs between models. Colours and numbers show signed class-wise mean concept projections for OlmoEarth and Prithvi-v2. Inset bars show normalized SHAP importance, with a full bar corresponding to a value of 0.5. Importance scores sum to one across the seven concepts for each class. Classes highlighted with red boxes illustrate (a) structural contrasts between broadleaf forest and cleared areas, (b) leaf-optical contrasts between oak and beech, and (c) the leaf area index interpretation examined for spruce and pine. Attributions are computed within each full classification task, rather than only between the displayed class pairs.
Spruce
Fir
Douglas fir
Pine
Larch
Number of samples
384
65
153
564
147
Mean NIR reflectance
17.9
24.4
23.5
23.3
24.7
TABLE II: Spruce has the lowest mean near-infrared reflectance (%) among the conifer classes. Reflectance is averaged over test samples for each class. The low spruce reflectance does not necessarily indicate low leaf area index; it could instead reflect crown self-shadowing due to its typically dense foliage, confounding interpretation.
Earth-observation (EO) foundation models have become exceptionally effective at learning se mantic, high-dimensional geospatial embeddings, while modern weather and climate models have demonstrated that Earth-specific geometry, spherical operators, meshes, and hybrid physical solvers can materially improve prediction. Yet these two advances are not equivalent. A conventional latent embedding has no inherent physical transformation law, whereas scalar fields, tangent polar-vector fields, axial/pseudovector quantities, covectors, and higher-order tensors transform differently under rotations, reflections, and changes of local coordinate frame. This proposal asks whether a general purpose Earth foundation model should preserve those distinctions explicitly, or whether standard embeddings plus augmentation already learn everything that matters. The central contribution is therefore not a more complicated architecture by assumption, but a staged falsification program. A compute-conscious ERA5 dry run first compares conventional, augmentation-matched, typed equivariant, and Hodge/Helmholtz variants under spatial, temporal, orientation, and low-data shifts. Only if explicit geometric typing yields reproducible improvements does the program advance toward a multimodal Earth foundation model in which semantic embeddings coexist with physically typed fields. The proposed gap is narrower and more defensible than claiming that current models ignore geometry entirely: several systems already respect spherical domain geometry, and emerging work explicitly learns scalar/vector fields on spheres. The unresolved question is whether foundation-scale, multimodal, parity-aware field typing produces practical gains beyond those existing approaches.
Foundation models (FMs) have transformed machine learning from isolated task-specific model development toward general-purpose models pretrained on broad data and adapted to multiple downstream tasks. Earth observation (EO) is an important domain for this paradigm because satellite and airborne archives are large, high-revisit, and increasingly multimodal, while reliable field labels are often sparse. Remote sensing foundation models (RSFMs) cannot be transferred reliably/optimally without domain-specific adaptation. This is because EO data are governed by measurement physics and operational decision constraints. This chapter reviews the design principles arising from these domain-specific constraints. It first defines the FMs paradigm in remote sensing (RS), then synthesizes the current model landscape, pretraining objectives, architecture designs, downstream adaptation and trustworthiness requirements. The chapter also incorporates recent benchmark evidence showing that no single geospatial foundation model is universally best and that inconsistent evaluation remains a major issue to fair comparison and reliable deployment. In addition, two brief environmental monitoring case studies; physics-informed spectral targeted masking for harmful algal bloom prediction and reinforcement learning for adaptive environmental monitoring station selection to illustrate the FMs domain-guided principles in practice. This chapter posits that next-generation RSFMs should be evaluated not only by benchmark accuracy, but also by modality-aware transfer and physically plausible representations for trustworthy EO decisions.
Syed Usama Imtiaz, Mitra Nasr Azadani, Nasrin Alamdari
Department of Civil and Environmental Engineering, Florida State University
Earth-observation (EO) foundation models provide reusable embeddings, yet downstream task accuracy does not reveal whether these representations encode geographic information, which may be beneficial for location-aware applications but potentially detrimental when representations invariant to geographic location are desired. We therefore evaluate the geographic coordinate robustness of Tessera v1, Tessera v1.1, and AlphaEarth by testing whether coordinates can be predicted from the embedding representations using 284 quality-verified European solar farms from 2024. We assessed geographic information content information through the association between cosine and geodesic distances and through prediction of projected coordinates in EPSG:3035. Embeddings from all three EO foundation models contain recoverable geographic information. All prediction models significantly outperform training-range uniform random sampling baselines, with AlphaEarth exhibiting the strongest distance association and lowest mean geodesic error. Both Tessera variants also yielded higher geographic distance correlations than the Sentinel-2 controls. These findings motivate geographic information content as an additional criterion for auditing EO foundation models.
Peiwen Zhang, Kristie Hu, Jovana Knezevic +2
University of Waterloo · University of Cambridge · Aalto University