SPAR: Placement-aware Self-Supervised Representation Learning for Distributed Sensing
Organizations: University of Illinois Urbana-Champaign, Urbana, IL, USA · Stanford University, Stanford, CA, USA
Abstract
To support distributed sensing applications that require understanding of entities in the environment and their locations, this paper develops a self-supervised approach for placement-aware representation learning in spatially-distributed multi-sensor systems. The goal is to produce a latent representation of the physical environment that decouples upstream sensor placement from downstream analytics (in the sense that the pretrained encoder is reused without retraining when sensor placement changes), while supporting spatial reasoning tasks, such as joint target localization and classification, that are sensitive to sensor placement. A challenge lies in the multiplicity of sensing vantage points that collectively offer information on the shared physical environment. The dependence of measurements on sensor locations implies that the latent representation must jointly encode both the observed signals and observer vantage points, doing so in a way that supports the downstream tasks. Moreover, since the exact deployment-time sensor placement is generally not known at training time, the representation learning framework must be pre-trained to produce such a representation without (training-time) knowledge of deployment-time placement. This challenge distinguishes our goal from approaches, such as NeRF and Gaussian Splatting, that are trained to represent each particular environment given a specific placement of observation points in that environment. In contrast, our representation learning solution, called SPAR, pretrains an encoder independently of future placement by utilizing a novel training objective that emphasizes the duality between signals and vantage points...
Figures & tables
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
| Site ID | Terrain | Weather | Targets | # Nodes | Length |
|---|---|---|---|---|---|
| H08 | Asphalt & gravel | Sunny | C, G, M, X | 6 | 2.77 h |
| H24 | Asphalt & gravel | Rainy | C, G, M, X | 6 | 3.43 h |
| S31 | Dirt & gravel | Sunny | C, G, M, X | 6 | 2.78 h |
| A06 | Asphalt | Sunny | C, G, X | 6 | 2.14 h |
| I29 | Concrete | Windy | C, G, M, X | 8 | 4.14 h |
| I22 | Concrete | Sunny | C, M, X | 8 | 3.00 h |
| Notation | Dimension(s) | Definition |
|---|---|---|
| Number of modalities | ||
| Number of sensors | ||
| Number of tokens | ||
| Number of visible tokens | ||
| Model dimension, tokenized signal dimension | ||
| Spatial-position dimension |
Representation Learning
Representation learning develops data transformations that expose useful structure for prediction, transfer, generation, and control.
See all 860 papers