Semantic Privacy Protection with Utility Preservation for 3D Point Clouds

Jinchang zhang; Jiakai Lin; David Crandall; Guoyu Lu

cs.CVSep 12, 2026

Semantic Privacy Protection with Utility Preservation for 3D Point Clouds

Authors: Jinchang zhang, Jiakai Lin, David Crandall, Guoyu Lu

Organizations: Department of Computer Science, Indiana University, Bloomington, IN, 47405, USA.

Abstract

Point cloud data face serious semantic privacy risks during acquisition, transmission, and cross-institutional sharing. Existing methods mostly rely on geometric perturbation or destructive encryption, which can reduce the recognizability of the original class but often impair downstream usability. This paper proposes a class-transfer-based semantic encryption framework for point clouds, aiming to conceal original class information while preserving task utility and supporting authorized recovery. Specifically, we construct a unified latent space with a shared-backbone Normalizing Flow, and combine LoRA and FiLM to achieve parameter-efficient class-conditional adaptation. We further introduce diffusion-guided flow alignment to regularize the latent distribution, construct an energy-based category transition graph, and obtain an optimal class-transfer table through global matching. Then, a latent-space Neural ODE continuously evolves source-class latents into target-class latents, which are decoded into target-class point clouds through the inverse flow. We adopt attacker-oriented metrics, including New-Class Recognition Rate (NCRR), Original-Class Leakage Rate (OCLR), and Original Label Recovery Rate (OLRR), to evaluate privacy and utility. Experiments on classification and segmentation benchmarks show that the proposed method achieves controllable semantic transformation, effectively reduces original-class semantic leakage, preserves downstream learnability in the protected domain, and supports reliable authorized reconstruction.

Figures & tables

Figure 1: Overview of the framework. First, the input point cloud is encoded by PointNet++ into point cloud features, which are fed into a shared-weight Flow model together. Under the modulation of class embeddings, LoRA and FiLM jointly provide dual-level scale control, yielding the source latent representation. In the latent space, the corresponding target class is determined by the category transition table, and a neural ODE is used to continuously evolve the source latent into the target latent representation. Finally, the target latent is decoded into a target-class point cloud via the inverse target Flow, achieving semantic-level point cloud encryption and class-wise transformation.

Figure 2: Comparison between the proposed parameter-efficient conditional Flow and the standard Glow model. The proposed model (left) freezes the shared Flow backbone and introduces dual modulation for lightweight adaptation across classes: LoRA performs parameter-level low-rank fine-tuning, while FiLM provides feature-level semantic modulation. In contrast, the standard Glow (right) trains an independent Flow model for each class.

Dataset	Method	PointNet Qi et al. (2017a)	PointNet++ Qi et al. (2017b)	DGCNN Wang et al. (2019)	PointCNN Li et al. (2018)	GBNet Qiu et al. (2022)	Point Trans Zhao et al. (2021)	PointMLP Ma et al. (2022)
ScanObjectNN	Acc	0.682 / 0.664	0.779 / 0.759	0.781 / 0.771	0.785 / 0.775	0.805 /0. 785	0.828 / 0.808	0.854 / 0.840
	NCRR	0.651	0.746	0.748	0.752	0.772	0.794	0.820
	OCLR	0.172	0.120	0.119	0.117	0.106	0.094	0.080
	OLRR	0.198	0.146	0.141	0.138	0.126	0.112	0.098
ModelNet40	Acc	0.892 / 0.881	0.909 / 0.903	0.929 / 0.918	0.925 / 0.914	0.938 / 0.927	0.929 / 0.918	0.941 / 0.929
	NCRR	0.850	0.860	0.875	0.872	0.885	0.876	0.890

Table 1: Comparison of classification performance on the ScanObjectNN Uy et al. (2019) and ModelNet40 datasets Wu et al. (2015) . In the acc column, the value before the slash represents the accuracy on the original data, while the value after the slash indicates the acc obtained by training the model on the encrypted (category-transformed) data.

Figure 3: Visual point clouds of class transfer on the ScanObjectNN Uy et al. (2019) . Each triplet shows, from left to right: the target, the source, and the result after encryption

Method	aero	bag	cap	car	chair	aerphone	guitar	knife	lamp	laptop	motor-bike	mug	pistol	rocket	skateboard	table	Mean
PointNet	83.4/81.53	78.7/80.0	82.5/84.27	74.9/73.18	89.6/87.83	73.0/71.19	91.5/89.97	85.9/84.06	80.8/82.51	95.3/93.99	65.2/63.31	93.0/94.47	81.2/79.34	57.9/56.48	72.8/71.02	80.6/78.71	81.2/79.95
PointNet++	82.3/80.93	79.0/77.57	87.7/86.46	77.3/75.52	90.8/92.05	71.8/70.79	91.0/89.88	85.0/83.03	83.7/84.86	95.3/94.05	71.6/70.38	94.1/92.9	81.3/80.0	58.7/56.75	76.4/75.0	82.6/84.38	82.7/81.47
Kd-Net	74.3/73.28	73.3/72.27	70.3/69.29	88.6/87.06	88.6/87.13	71.9/70.29	90.2/88.86	87.2/86.19	81.0/79.94	94.9/93.86	57.4/56.34	86.7/85.67	78.1/76.31	51.0/49.0	82.3/84.08	80.3/78.54	78.4/77.05
SO-Net	88.0/86.21	81.1/79.29	88.0/86.16	90.6/88.85	90.6/89.44	73.5/72.5	90.7/88.78	83.9/81.97	83.9/82.27	94.8/92.99	69.1/67.75	92.4/91.33	80.9/79.81	53.1/51.28	72.9/71.35	83.0/81.11	82.6/80.98
PCNN	85.5/84.32	79.5/78.1	79.5/77.78	73.2/72.0	90.9/89.55	73.2/71.3	90.3/88.9	90.0/88.04	86.9/85.45	95.8/93.9	69.0/67.04	92.1/90.64	86.3/84.5	51.0/49.88	83.1/81.55	83.3/85.14	82.5/80.96
DGCNN	86.7/85.21	77.8/75.93	86.7/88.35	77.8/76.6	90.6/88.74	74.7/72.98	91.2/89.49	87.5/86.32	86.8/85.01	95.7/94.35	66.3/64.95	81.1/82.85	81.1/79.63	63.5/62.17	74.5/75.76	82.6/81.36	82.0/80.91

Table 2: Segmentation results on the ShapeNetPart dataset Chang et al. (2015) .

Method	PointNet Qi et al. (2017a)	PointNet++ Qi et al. (2017b)	DGCNN Wang et al. (2019)	PointCNN Li et al. (2018)	GBNet Qiu et al. (2022)	Point Trans Zhao et al. (2021)	PointMLP Ma et al. (2022)
Clean data	0.682	0.779	0.781	0.785	0.805	0.828	0.854
optical chaotic Liu et al. (2023) $\dagger$	0.041/0.081	0.048/0.084	0.052/0.087	0.056/0.089	0.062/0.092	0.070/0.096	0.079/0.099
hybrid key Li et al. (2024) $\dagger$	0.214/0.272	0.258/0.284	0.271/0.296	0.286/0.307	0.294/ 0.321	0.306/0.336	0.314/0.349
UMT Wang et al. (2024) (Class-wise test set)	0.421/0.852	0.438/0.871	0.447/0.884	0.455/0.893	0.466/0.907	0.481/ 0.924	0.493/0.939
UMT Wang et al. (2024) ( Permuted-wise test set)	0.421/0.214	0.438/0.232	0.447/0.246	0.455/0.257	0.466/0.281	0.481/0.307	0.493/0.329
Ours	0.071 / 0.664	0.078 / 0.759	0.084 / 0.771	0.089 / 0.775	0.098 /0.785	0.109/ 0.808	0.119/ 0.840

Table 3: Comparison of different encryption methods in the original and encrypted domains on the ScanObjectNN Uy et al. (2019) . For each entry, the value before “/” denotes the classification accuracy when the classifier is trained on the original dataset and tested on the encrypted dataset; the value after “/” denotes the classification accuracy when the classifier is both trained and tested within the same protected/encrypted domain.

\dagger

indicates results implemented by ourselves.

Setting	ScanObjectNN												ModelNet40
	PointNet				PointNet++				PointMLP				PointNet				PointNet++				PointMLP
	A	N	O	R	A	N	O	R	A	N	O	R	A	N	O	R	A	N	O	R	A	N	O	R
1024	66.41	0.651	0.172	0.301	75.95	0.746	0.120	0.204	84.02	0.820	0.080	0.116	88.10	0.850	0.090	0.162	90.30	0.860	0.085	0.145	92.90	0.890	0.065	0.094
768	65.20	0.631	0.184	0.321	74.80	0.728	0.132	0.223	83.10	0.805	0.089	0.129	87.30	0.839	0.098	0.173	89.60	0.850	0.092	0.156	92.10	0.881	0.071	0.103
512	63.40	0.602	0.201	0.349	72.90	0.701	0.149	0.247	81.20	0.781	0.103	0.149	85.90	0.821	0.111	0.190	88.20	0.836	0.103	0.171	90.90	0.868	0.081	0.118
clean	66.41	0.651	0.172	0.301	75.95	0.746	0.120	0.204	84.02	0.820	0.080	0.116	88.10	0.850	0.090	0.162	90.30	0.860	0.085	0.145	92.90	0.890	0.065	0.094

Table 4: Robustness analysis under varying point densities and noise levels. A: Acc, N: NCRR, O: OCLR, R: OLRR.

Setting	box				display				door
	A	N	O	R	A	N	O	R	A	N	O	R
Full Joint Training	75.8	0.742	0.123	0.208	74.9	0.731	0.129	0.217	76.2	0.748	0.118	0.201
Incremental Adaptation	74.2	0.721	0.136	0.226	73.1	0.708	0.145	0.239	74.8	0.728	0.132	0.221

Table 5: Incremental adaptation results on unseen classes of ScanObjectNN using PointNet++. We treat box , display , and door as unseen categories. A: Acc, N: NCRR, O: OCLR, R: OLRR.

Method	Params (M)	NCRR $\uparrow$	OCLR $\downarrow$	Acc $\uparrow$
15 $\times$ Flow	159.75	0.71	0.18	0.8554
Shared + LoRA	20.51	0.75	0.12	0.8769
Shared + LoRA + FiLM	21.12	0.81	0.08	0.9032

Table 6: Comparison of parameter counts and privacy/utility metrics for different Flow models on the ModelNet40 datasets Wu et al. (2015) .

Figure 4: Energy and effectiveness of the class-transfer table on the ScanObjectNN dataset. (a) Energy matrix heatmap: each entry $E_{ij}$ denotes the conversion energy from source class $i$ to target class $j$ (lighter color = lower energy). (b) Source $\rightarrow$ target pairs obtained by energy-based one-to-one matching; thicker lines indicate lower energy (better pairs). (c) Confusion matrix after applying the transfer table. (d) Confusion matrix under random mapping.

Figure 5: Qualitative comparison of encryption–decryption reconstruction fidelity across different backbones. Red points denote the original point clouds before encryption, while blue points denote the decrypted point clouds.

Method	Params (M)	NCRR $\uparrow$	OCLR $\downarrow$	Acc $\uparrow$
Shared + LoRA ( $r=4$ )	15.58	0.74	0.13	0.8720
Shared + LoRA ( $r=8$ )	20.51	0.75	0.12	0.8769
Shared + LoRA ( $r=16$ )	30.37	0.76	0.11	0.8785
Shared + LoRA (no FiLM)	20.51	0.75	0.12	0.8769
Shared + LoRA + FiLM (last 10 blks)	20.82	0.79	0.09	0.8990
Shared + LoRA + FiLM (all blks)	21.12	0.80	0.08	0.9032

Table 7: Ablation studies on LoRA rank and FiLM placement under the shared Flow setting.

Figure 6: Comparison of latent trajectories with and without energy constraint. The figure visualizes class transitions from Class A to B in latent space. The energy-constrained flow (blue curve) evolves smoothly along low-energy valleys, while the unconstrained flow (orange path) exhibits irregular and unstable motion across high-energy regions.

Setting	NCRR $\uparrow$	OCLR $\downarrow$	Acc $\uparrow$
w/o Diffusion	0.78	0.11	0.8910
Diffusion	0.79	0.09	0.8985
Diffusion + $\mathcal{L}_{\text{align}}$	0.81	0.08	0.9032
Scale	0.77	0.11	0.8875
Scale + $\mathcal{L}_{\text{contrast}}$	0.79	0.09	0.8984
Scale + $\mathcal{L}_{\text{contrast}}$ + $\mathcal{L}_{\text{disp}}$	0.81	0.08	0.9032

Table 8: Ablation studies on the diffusion module and dual-level scale control.

$\mathcal{L}_{\text{align}}$	$\mathcal{L}_{\text{rev}}$	$\mathcal{L}_{\text{energy}}$	NCRR $\uparrow$	OCLR $\downarrow$	Acc $\uparrow$
			0.78	0.11	0.8920
✓			0.79	0.10	0.8965
✓	✓		0.80	0.09	0.9008
✓	✓	✓	0.81	0.08	0.9032

Table 9: Ablation study on the ODE-related losses. distribution alignment loss

\mathcal{L}_{\text{align}}

, reversibility consistency loss

\mathcal{L}_{\text{rev}}

, and local energy smoothness loss

\mathcal{L}_{\text{energy}}

Flow	Diffusion	Dual	Cate	Trans Flow	NCRR $\uparrow$	OCLR $\downarrow$	Acc $\uparrow$
					0.71	0.18	0.8554,
✓					0.74	0.14	0.8695
✓	✓				0.75	0.12	0.8769
✓	✓	✓			0.76	0.12	.8830.
✓	✓	✓	✓		0.78,	0.10	0.8905
✓	✓	✓	✓	✓	0.81	0.08	0.9032

Table 10: Ablation study on the ModelNet40 dataset Wu et al. (2015) . Dual: DDual-Level Scale Control. Cate: Category Transition Table. Trans Flow: Category Transition Flow.

Figure 7: Sensitivity analysis of the loss weights. For (a) and (b), we vary $\lambda_{\text{align}}$ and $\lambda_{\text{disp}}$ from 0.01 to 0.1. For (c) and (d), we vary $\lambda_{\text{rev}}$ and $\lambda_{\text{energy}}$ within $[0.1,10]$ , while keeping the other loss weights fixed.

Figure 8: Sensitivity of the three module loss weights. We vary one weight within $[0.01,0.40]$ while fixing the other two at their optimal values. The left figure corresponds to the diff module loss, the middle figure corresponds to the dual module loss, and the right figure corresponds to the ODE module loss.

Privacy-Preserving ML

Privacy-preserving machine learning develops methods to learn, share, and run models while limiting exposure of sensitive data.

See all 423 papers

COSTA: A Cluster-Centric Paradigm for Annotation-Free Open-Set Semantic Segmentation of Aerial Point Clouds with Domain Shifts
Aug 19, 2026Yanghong Lin, Li Fang, Tianyu Li +2Graph-Based Label Propagation 3D Point Cloud Segmentation
Fujian Institute of Research on the Structure of Matter, Chinese Academy of Sciences, Fuzhou, 350108, Fujian, China. · Spatial Intelligence and Urban Computing, Institute of Urban Environment, Chinese Academy of Sciences, Xiamen, 361021, Fujian, China. · State Key Lab for Ecological Security of Regions and Cities, Institute of Urban Environment, Chinese Academy of Sciences, Xiamen, 361021, Fujian, China. +2
Task-Oriented Wireless Transmission of 3D Point Clouds: Geometric Versus Semantic Robustness
Mar 13, 2026Vukan Ninkovic, Tamara Sobot, Vladimir Vincan +3Semantic Communication Point Cloud Understanding
The Institute for Artificial Intelligence Research and Development of Serbia, Serbia · Faculty of Technical Sciences, University of Novi Sad, Serbia
APC: Transferable and Efficient Adversarial Point Counterattack for Robust 3D Point Cloud Recognition
Apr 17, 2026Geunyoung Jung, Soohong Kim, Inseok Kong +1Point Cloud Classification Adversarial Robustness
Department of Artificial Intelligence, University of Seoul, South Korea · Department of Geo Informatics, University of Seoul, South Korea

Aug 19, 2026J/K move · Enter open · S save

COSTA: A Cluster-Centric Paradigm for Annotation-Free Open-Set Semantic Segmentation of Aerial Point Clouds with Domain Shifts

Yanghong Lin, Li Fang, Tianyu Li, Shudong Zhou+1

Fujian Institute of Research on the Structure of Matter, Chinese Academy of Sciences, Fuzhou, 350108, Fujian, China. · Spatial Intelligence and Urban Computing, Institute of Urban Environment, Chinese Academy of Sciences, Xiamen, 361021, Fujian, China. · State Key Lab for Ecological Security of Regions and Cities, Institute of Urban Environment, Chinese Academy of Sciences, Xiamen, 361021, Fujian, China.+2

Graph-Based Label Propagation 3D Point Cloud Segmentation Semantic Segmentation 3D Semantic Segmentation Test-Time Adaptation

Open paper AskPDF arXiv

Semantic segmentation of aerial point cloud is trapped in a generalization crisis under distinct domain shifts. While test-time adaptation offers a privacy-preserving and computationally efficient way to adapt pre-trained models to unlabeled target-domain data during inference, existing methods, bound to closed-set label assumptions and non-scalable point-wise segmentation pipelines, still struggle with semantic shifts. We ask: can we adapt any given pre-trained aerial point cloud segmentation model to a shifted target domain at the inference phase alone, without additional training, while segmenting target-specific categories beyond the source label space on demand? This paper introduces COSTA, which breaks this limitation by shifting from closed-set point-wise adaptation to cluster-centric open-set semantic propagation. Our core discovery is that, once effectively adapted at test time, the rich feature distribution of aerial point clouds can be distilled into a compact set of well-separated semantic centroids that are transferable across label spaces. COSTA leverages this to reformulate open-set semantic segmentation as a cluster-level propagating process: it first bridges the domain gap through proven test-time adaptation, then groups each batch of target-domain points into a small set of semantic clusters based on the similarity distribution in the adapted feature space, and finally propagates high-confidence pseudo labels obtained from an open-vocabulary vision-language model to all points through cluster-level voting. This cluster-centric paradigm enables test-time adaptation of aerial point clouds under significant domain gaps with mixed semantic shifts. With DALES as the source domain, COSTA enables on-demand segmentation across three aerial point cloud benchmarks with distinct domains and heterogeneous category spaces, achieving up to 70.09% mIoU under this new setting.

Abstract

Privacy-Preserving ML

Explore similar work

COSTA: A Cluster-Centric Paradigm for Annotation-Free Open-Set Semantic Segmentation of Aerial Point Clouds with Domain Shifts

Task-Oriented Wireless Transmission of 3D Point Clouds: Geometric Versus Semantic Robustness

APC: Transferable and Efficient Adversarial Point Counterattack for Robust 3D Point Cloud Recognition