Multi-level latent-diffusion adaptation for cross-region point cloud segmentation of ancient terraces
Authors: Sina Mohammadi, Roderik Lindenbergh, Javier Ruiz-Pérez, Marco Madella, Sebastian Fajardo
Organizations: Leiden Institute of Advanced Computer Science (LIACS), Leiden University, Einsteinweg 55, 2333 CA, Leiden, The Netherlands · Geoscience & Remote Sensing Department, Delft University of Technology, Stevinweg 1, 2628 CN Delft, Delft, The Netherlands · Department of Humanities, Universitat Pompeu Fabra, carrer de la Mercé, 08002 Barcelona, Spain · ICREA, Pg. Lluís Companys 23, 08010 Barcelona, Spain · School of Geography, Archaeology, and Environmental Studies, University of the Witwatersrand, 1 Jan Smuts Avenue, Braamfontein 2000 Johannesburg, South Africa
Automated lidar feature detection from lidar point clouds can support archaeological surveys, but models trained in one landscape may transfer poorly to another because of differences in terrain, vegetation, acquisition characteristics, and feature morphology. We address this problem for cross-region detection of ancient terraces using transductive unsupervised domain adaptation that exploits an unlabelled target survey. We propose MELD (Multi-level Embedding-space Latent Diffusion), which trains separate diffusion denoisers on target-domain representations at four hierarchical PointNet++ feature levels. These target distribution priors are then used to refine source representations during supervised training and target representations during inference, allowing adaptation to act directly on the multi-scale features consumed by the segmentation decoder. We additionally investigate local geometric descriptors derived from K-nearest-neighbour neighbourhoods, including surface normals, slope, and aspect. MELD is evaluated bidirectionally between the Teyuna and Miraflores lidar surveys in the Sierra Nevada de Santa Marta, Colombia, against a source-only model and nine configurations of established domain adaptation approaches. MELD raises F1 over the source-only model from 0.397 to 0.459 for Teyuna-to-Miraflores transfer and from 0.557 to 0.594 in the reverse direction, achieving the highest F1 among the evaluated adaptation methods in both directions. Adding local geometry further raises F1 to 0.542 and 0.606, respectively. Multi-level ablation shows that refining all four PointNet++ levels provides the strongest performance in both directions. These results show that explicit modelling of target feature distributions in hierarchical point cloud feature space, complemented by local geometry, provides an effective route to cross-region archaeological mapping without target labels.
Figures & tables
Figure 1: Reference terrace annotations for the two study regions. Left: Teyuna (Ciudad Perdida); right: Miraflores. Reference terrace annotations are shown over the labelled study extents. Mapped terraces cover approximately 6.5% of the labelled Teyuna extent and approximately 1.5% of the Miraflores extent.
Figure 2: High-level overview of the MELD framework. During joint training, unlabeled target/test features train the level-specific diffusion denoisers, while labeled source features are refined by the target-trained denoisers and decoded under source ground-truth supervision. The diffusion loss updates the denoisers, whereas the segmentation loss updates PointNet++ through the straight-through refinement path. A held-out source-validation set ranks checkpoints by adapted validation F1. At inference, the top three paired PointNet++ denoiser checkpoints are applied independently to all target/test points using deterministic target-guided DDIM refinement; their terrace-class probabilities are averaged point-wise and thresholded at 0.5 to obtain the final binary prediction. No target labels are used for adaptation or checkpoint selection. Preprocessing and the PointNet++ backbone are detailed in Figure 3 , target domain latent diffusion adaptation in Figure 4 , and source-feature refinement with the straight-through estimator in Figure 5 ; adapted inference is described in Section 3.5 .
Figure 3: PointNet++ segmentation backbone. Each 20m×20m point cloud tile is converted by sampling or padding into a fixed 4096-point input patch, which is then normalized. The patch may contain either only xyz coordinates ( C=3 ) or xyz coordinates augmented with optional KNN-derived geometric features ( C=9 ), including normals, slope, and sine/cosine of aspect. The encoder comprises four set-abstraction (SA) layers that progressively reduce the number of points while increasing feature dimensionality, producing hierarchical latent features Z1 , Z2 , Z3 , and Z4 . The decoder then uses four feature-propagation (FP) stages with interpolation and skip connections to recover point-wise features at the original resolution. A per-point classification head, implemented with 1×1 convolutions, batch normalization, ReLU, dropout, and log-softmax, produces per-point class log-probabilities; applying exp(⋅) yields the corresponding per-point class probabilities. The generic Zl notation used in this figure denotes the multi-level encoder features that serve as latent representations in the target-adaptation and source-refinement stages, where they are written explicitly as Ztarl and Zsrcl in Figures 4 and 5 , respectively.
Figure 4: target domain latent diffusion adaptation. Unlabeled target domain input patches Xtar are passed through the shared PointNet++ encoder to obtain multi-level latent features Ztarl , l∈{1,2,3,4} . At each level, the target features are detached from the backbone and flattened into token matrices U0,tarl , so the target domain diffusion objective does not propagate gradients into PointNet++. Each token is independently assigned a diffusion timestep til and perturbed with Gaussian noise according to the forward diffusion process. A level-specific denoiser Dl(⋅,⋅;ϕl) , conditioned on the noisy token and its timestep embedding, predicts the added noise. Within each level, the same denoiser is shared across all tokens, while denoiser parameters are not shared between feature levels. The level-wise noise-prediction losses are computed as mean-squared error between the sampled and predicted noise and are combined into the total diffusion loss Ldiff . Consequently, target domain adaptation updates only the denoiser parameters ϕl , allowing the denoisers to learn the latent feature distribution of the unlabeled target domain for subsequent source-feature refinement and adapted inference. The figure shows the principal configuration in which all four PointNet++ feature levels are active; the level-ablation experiments use the corresponding subsets of these levels.
Figure 5: Source refinement for supervised segmentation training. A labeled source-domain input patch Xsrc is passed through the shared PointNet++ encoder to obtain hierarchical latent features Zsrcl for l∈{1,2,3,4} . At each feature level, the encoder output is tokenized into U0,srcl , partially noised to a shared starting depth t0l , and then refined token-wise through a deterministic DDIM reverse process using the level-specific target-trained denoiser Dl . The refined token matrix U0,srcl is unflattened to Zsrcl and supplied to the decoder through a straight-through estimator (STE), so that the decoder uses the refined features in the forward pass while segmentation gradients flow back to the original encoder features during backpropagation. The decoder output is supervised by the source ground-truth labels using the class-weighted NLL loss Lseg . The figure shows the principal configuration in which all four PointNet++ feature levels are refined. During training, source refinement is stochastic overall because the initial Gaussian perturbation is sampled randomly, whereas the subsequent DDIM reverse trajectory is deterministic conditional on the resulting partially noised token.
Method
Levels
Teyuna → Miraflores
Miraflores → Teyuna
Mean
Source-only PointNet++
—
0.397±0.037
0.557±0.011
0.477
MMD/DAN
single
0.422±0.010
0.540±0.018
0.481
MMD/DAN
multi
0.352±0.028
0.531±0.016
0.442
Deep CORAL
single
0.342±0.037
0.572±0.008
0.457
Deep CORAL
multi
0.382±0.034
0.573±0.027
0.477
DANN
single
0.395±0.102
0.551±0.016
0.473
Table 1: Cross-region terrace detection (point level, all points): F1 as mean ± standard deviation over three random seeds. Single applies adaptation at the deepest level ( l=4 ); multi at all four levels ( l∈{1,2,3,4} ). Mean is the unweighted average of the two directions. Best result per column in bold. The target-only oracle (italic) is an in-domain supervised reference trained with five-fold cross-validation on the evaluation region of each direction; it is not a competing UDA method and is reported for context only.
Active diffusion levels
Teyuna → Miraflores
Miraflores → Teyuna
{4}
0.408±0.037
0.564±0.007
{3,4}
0.443±0.019
0.534±0.025
{2,3,4}
0.440±0.057
0.586±0.030
{1,2,3,4}
0.459±0.038
0.594±0.015
Table 2: MELD level ablation: effect of activating diffusion adaptation at different PointNet++ feature levels (no KNN features; adaptation enabled). The first column lists the active level set. F1 as mean ± standard deviation over three random seeds. Best result per column in bold.
Figure 6: Precision–recall curves for the three principal configurations, averaged over three random seeds. The left panel shows Teyuna → Miraflores and the right panel Miraflores → Teyuna. Solid lines show the mean across three random seeds, and shaded regions indicate one standard deviation across seeds. The compared configurations are source-only PointNet++, MELD, and MELD+geometry.
3D point cloud segmentation assigns semantic, instance, or part labels to points for scene understanding, mapping, robotics, and scientific measurement.
We study domain generalization for LIDAR semantic segmentation. Deep neural networks achieve strong performance on 3D point clouds, yet real-world LIDAR exhibits sensor-specific sampling patterns that hinder transfer across datasets. In many applications, performance on completely unseen sensors is crucial. A model should preferably work on a new setup without prior access to its data or labels. To this end, we introduce a source-only training scheme that adds an occupancy prediction auxiliary objective. We aggregate source-domain scans into a simple world model and use it as the supervision signal, replacing single-scan standard segmentation training. A key property of our approach is that it does not alter the input scan. During training and testing the network consumes the same single-frame LIDAR input, only the supervision signal is enriched via the occupancy objective. At deployment, there is no runtime overhead as the auxiliary head can be dropped for pure segmentation, or kept to obtain scene completion alongside semantic segmentation. In an experimental evaluation across distinct LIDAR datasets, we demonstrate that a model's generalization to new domains can be significantly affected by how it is supervised. In certain setups, performance on unseen domains improves without any explicit domain adaptation (no target scans, no target labels, no sensor priors) and without prediction refinement. At the same time, our method maintains competitive source-domain segmentation performance and incurs no additional inference cost.
Christoph B Rist, Larissa T Triess, Markus Enzweiler +1
Technical University Delft, Intelligent Vehicles Group, The Netherlands · Hochschule Esslingen, Esslingen, Germany
Transferring the rich priors of large 2D foundation models to sparse 3D LiDAR remains challenging, as training native 3D foundation models at comparable scale is limited by data and annotation scarcity. We introduce a LiDAR-conditioned diffusion model trained on pseudo-labels from off-the-shelf 2D foundation models. The model supports multiple output modalities, including depth, semantic segmentation and instance prediction, selectable via a textual task prompt. Because the model is conditioned on LiDAR, both its outputs and its intermediate UNet features can be projected back onto the input point cloud, enabling analysis of a 3D representation learned entirely under 2D supervision. We study this representation directly in point-cloud space, explicitly excluding raw spatial coordinates to isolate feature content from projection geometry. Linear probes recover up to ~23% Mean Intersection over Union (MIoU) on 3D semantic classes, compared to ~3.5% for a matched Gaussian-noise control, indicating substantial non-trivial structure. Pairwise cosine similarity across modality-specific feature streams reveals a layered organization. Early encoder layers remain weakly aligned across modalities while individually decodable, intermediate layers converge toward a shared representation, and decoder layers re-specialize toward task-specific outputs. These findings indicate that LiDAR-conditioned diffusion models can induce structured 3D representations from 2D supervision alone, with a modality-dependent manifold that locally unifies near a shared bottleneck. This positions diffusion as a viable mechanism for transferring large-scale 2D priors into sparse 3D domains.
Samed Doğan, Nico Leuze, Alfred Schöttl
Department of Electrical Engineering and Information Technology Munich University of Applied Sciences Munich, Bavaria 80335
Semantic segmentation of aerial point cloud is trapped in a generalization crisis under distinct domain shifts. While test-time adaptation offers a privacy-preserving and computationally efficient way to adapt pre-trained models to unlabeled target-domain data during inference, existing methods, bound to closed-set label assumptions and non-scalable point-wise segmentation pipelines, still struggle with semantic shifts. We ask: can we adapt any given pre-trained aerial point cloud segmentation model to a shifted target domain at the inference phase alone, without additional training, while segmenting target-specific categories beyond the source label space on demand? This paper introduces COSTA, which breaks this limitation by shifting from closed-set point-wise adaptation to cluster-centric open-set semantic propagation. Our core discovery is that, once effectively adapted at test time, the rich feature distribution of aerial point clouds can be distilled into a compact set of well-separated semantic centroids that are transferable across label spaces. COSTA leverages this to reformulate open-set semantic segmentation as a cluster-level propagating process: it first bridges the domain gap through proven test-time adaptation, then groups each batch of target-domain points into a small set of semantic clusters based on the similarity distribution in the adapted feature space, and finally propagates high-confidence pseudo labels obtained from an open-vocabulary vision-language model to all points through cluster-level voting. This cluster-centric paradigm enables test-time adaptation of aerial point clouds under significant domain gaps with mixed semantic shifts. With DALES as the source domain, COSTA enables on-demand segmentation across three aerial point cloud benchmarks with distinct domains and heterogeneous category spaces, achieving up to 70.09% mIoU under this new setting.
Yanghong Lin, Li Fang, Tianyu Li +2
Fujian Institute of Research on the Structure of Matter, Chinese Academy of Sciences, Fuzhou, 350108, Fujian, China. · Spatial Intelligence and Urban Computing, Institute of Urban Environment, Chinese Academy of Sciences, Xiamen, 361021, Fujian, China. · State Key Lab for Ecological Security of Regions and Cities, Institute of Urban Environment, Chinese Academy of Sciences, Xiamen, 361021, Fujian, China. +2