Adapting Point Cloud Analysis via Multimodal Bayesian Distribution Learning

Xingyu Zhu; Liang Yi; Shuo Wang; Wenbo Zhu; Yonglinag Wu; Beier Zhu; Hanwang Zhang

cs.CVMar 23, 2026

Adapting Point Cloud Analysis via Multimodal Bayesian Distribution Learning

Authors: Xingyu Zhu, Liang Yi, Shuo Wang, Wenbo Zhu, Yonglinag Wu, Beier Zhu, Hanwang Zhang

Organizations: MoE Key Lab of BIPC, University of Science and Technology of China · Opus AI Research · Southeast University · Nanyang Technological University

Abstract

Multimodal 3D vision-language models show strong generalization across diverse 3D tasks, but their performance still degrades notably under domain shifts. This has motivated recent studies on test-time adaptation (TTA), which enables models to adapt online using test-time data. Among existing TTA methods, cache-based mechanisms are widely adopted for leveraging previously observed samples in online prediction refinement. However, they store only limited historical information, leading to progressive information loss as the test stream evolves. In addition, their prediction logits are fused heuristically, making adaptation unstable. To address these limitations, we propose BayesMM, a Multimodal Bayesian Distribution Learning framework for test-time point cloud analysis. BayesMM models textual priors and streaming visual features of each class as Gaussian distributions: textual parameters are derived from semantic prompts, while visual parameters are updated online with arriving samples. The two modalities are fused via Bayesian model averaging, which automatically adjusts their contributions based on posterior evidence, yielding a unified prediction that adapts continually to evolving test-time data without training. Extensive experiments on multiple point cloud benchmarks demonstrate that BayesMM maintains robustness under distributional shifts, yielding over 4% average improvement.

Figures & tables

Figure 1: Comparison between cache-based adaptation and our distribution-based modeling. (a) Cache-based methods rely on discrete memory updates, storing a small number of recent samples in a fixed-size cache. (b) Our BayesMM models class-wise distributions across modalities rather than individual samples.

Figure 2: Comparison of distribution consistency across adaptation steps. The Kullback–Leibler (KL) divergence and Maximum Mean Discrepancy (MMD) are measured at different time steps during test-time adaptation.

Figure 3: Overview of the proposed BayesMM framework. A frozen point cloud encoder extracts geometric features from streaming inputs, and a frozen language model provides textual embeddings as semantic priors. Both modalities are represented by Gaussian distributions, where geometric ones are updated online with incoming samples. Bayesian weighting fuses the two modalities into a unified posterior for adaptive and training-free point cloud recognition.

Method	ModelNet40	S-PB_RS_T50	O-LVIS	Omni3D			Avg.
Method	(10000 pts)	(2048 pts)	(10000 pts)	(1024 pts	4096 pts	16384 pts)	Avg.
ULIP [ 38 ]	58.75	46.44	6.24	8.39	7.75	7.28	22.48
+ Point-Cache (Global)	61.22	50.21	7.02	10.00	9.36	8.43	24.37
+ Point-Cache (Hierarchical)	62.93	51.80	7.02	10.47	9.75	8.90	25.15
+ BayesMM	67.13	53.67	7.79	11.27	10.68	9.47	26.67
ULIP-2 [ 39 ]	72.97	47.13	30.26	26.36	29.20	26.58	38.75

Table 2: Recognition accuracy comparison on multiple benchmarks. S-PB_RS_T50 denotes the hardest split of ScanObjectNN. O-LVIS and Omni3D refer to Objaverse-LVIS and OmniObject3D, respectively. The number under each dataset indicates the number of points per object (pts). In Omni3D, each object may contain a variable number of points.

Method	MN_11 $\rightarrow$ SONN_11			SN_9 $\rightarrow$ SONN_9			Avg.
Method	OBJ	OBJ_BG	PB_T50_RS	OBJ	OBJ_BG	PB_T50_RS	Avg.
ULIP [ 38 ]	57.05	50.32	32.60	61.00	61.00	44.38	51.06
+ Point-Cache (Global)	62.32	52.63	34.97	65.50	62.50	47.36	54.21
+ Point-Cache (Hierarchical)	64.42	56.63	35.77	67.25	64.50	47.61	56.03
+ BayesMM	65.41	60.42	45.34	67.25	68.50	55.36	60.38
ULIP-2 [ 39 ]	50.94	52.42	39.12	51.50	59.25	46.35	49.93

Table 3: Recognition accuracy comparison on Sim-to-Real. Two evaluation settings are considered: MN_11

\rightarrow

SONN_11 and SN_9

\rightarrow

SONN_9. The dataset on the left side of

\rightarrow

stands for simulated data, while the dataset on the right side indicates real-world data. 11 classes are shared between MN_11 and SONN_11, while 9 classes are common between SN_9 and SONN_9. In the experiments, each point cloud is represented by 2,048 points. MN: ModelNet, SN: ShapeNet.

	Geometric Distribution	Texutal Distribution	Bayes Weighting	ULIP2 [ 39 ]
	Geometric Distribution	Texutal Distribution	Bayes Weighting	S-OBJ	Omni3D
(1)	✗	✗	✗	42.00	26.58
(2)	✔	✗	✗	46.47	26.63
(3)	✗	✔	✗	52.50	30.79
(4)	✔	✔	✔	53.02	31.12

Table 4: Ablation study on the components of BayesMM . S-OBJ denotes the OBJ_ONLY split of ScanObjectNN.

Panel of Figure 4 — Figure 4: Hyperparameter sensitivity of BayesMM with respect to the geometric prior $\alpha^{2}$ and textual prior $\beta^{2}$ , evaluated on ULIP and ULIP-2 backbones.

Panel of Figure 5 — Figure 5: Analysis of geometric and textual weights in BayesMM across different backbones on the ModelNet-C dataset.

Method	ModelNet-C	Omni3D	O-LVIS	#Params
Method	(40)	(216)	(1156)	(M)
ULIP [ 38 ]	1,556	1,558	1,556	85.7
+ Point-Cache	1,556	1,558	1,566	85.7
+ BayesMM	1,560	1,560	1,562	85.7
Uni3D [ 48 ]	5,062	5,062	5,062	1016.5
+ Point-Cache	5,064	5,068	5,090	1016.5

Table 5: Memory usage (MB) comparison across different datasets. Numbers below each dataset name indicate the number of classes. Point-Cache denotes the hierarchical variant.

Method	ULIP	ULIP2	OpenShape	Uni3D
Vanilla	11.35	12.82	8.68	7.77
+ Point-Cache [ 24 ]	11.27	12.75	8.60	7.69
+ BayesMM	10.99	12.45	8.29	7.45

Table 6: Inference throughput (samples per second) on the ModelNet40. All experiments are conducted with a batch size of 1 on an RTX 3090 GPU. Point-Cache denotes the hierarchical variant.

Panel of Figure 6 — Figure 6: t-SNE visualization of feature distributions derived from classifiers constructed under different strategies. t-SNE visualization of feature distributions obtained from classifiers built with different learning strategies.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Original Data	Corruption Type							Avg.
Method	SONN	Add Global	Add Local	Drop Global	Drop Local	Rotate	Scale	Jitter	Avg.
ULIP [ 38 ]	49.05	31.50	34.77	51.29	38.38	48.36	44.58	36.83	41.85
+ Point-Cache (Global)	52.15	35.80	37.01	54.39	41.82	49.74	45.09	40.28	44.54
+ Point-Cache (Hierarchical)	52.15	32.01	38.04	54.56	45.27	50.95	45.96	39.24	44.77
+ BayesMM	54.04	39.24	38.04	55.42	46.30	52.15	47.68	35.11	45.75
ULIP-2 [ 39 ]	42.00	40.45	41.31	37.69	30.29	38.21	44.45	22.89	37.16

Appendix

Table G : Comparison of recognition accuracy on S-OBJ_ONLY-C, which contains seven types of corruptions. Results are reported at corruption severity level 2. Each clean point cloud contains 1024 points. SONN refers to ScanObjectNN.

Method	Original Data	Corruption Type							Avg.
Method	SONN	Add Global	Add Local	Drop Global	Drop Local	Rotate	Scale	Jitter	Avg.
ULIP [ 38 ]	45.96	27.19	25.82	45.61	34.25	40.96	40.10	30.98	36.36
+ Point-Cache (Global)	48.88	30.46	30.46	49.05	39.59	44.92	42.17	31.84	39.68
+ Point-Cache (Hierarchical)	49.74	28.23	30.12	48.71	40.45	43.55	40.28	34.42	39.44
+ BayesMM	52.67	33.05	34.08	50.43	41.65	48.53	45.78	31.50	41.81
ULIP-2 [ 39 ]	48.19	40.62	38.90	39.24	32.36	41.14	42.86	21.17	38.04

Appendix

Table H : Comparison of recognition accuracy on S-OBJ_BG-C, which includes seven types of corruptions. Results are reported at corruption severity level 2. Each clean point cloud contains 1024 points.

Method	Original Data	Corruption Type							Avg.
Method	SONN	Add Global	Add Local	Drop Global	Drop Local	Rotate	Scale	Jitter	Avg.
ULIP [ 38 ]	29.29	19.26	18.39	30.99	23.91	27.48	26.34	21.44	24.64
+ Point-Cache (Global)	32.37	22.87	20.85	33.31	27.90	30.85	28.63	24.53	27.66
+ Point-Cache (Hierarchical)	32.48	23.46	22.69	34.70	31.75	33.00	28.28	25.05	28.93
+ BayesMM	40.52	29.53	25.92	39.21	33.59	35.74	32.44	24.67	33.18
ULIP-2 [ 39 ]	33.38	30.29	29.42	28.24	24.91	28.56	30.22	12.98	27.25

Appendix

Table I : Comparison of corruption generalization on S-PB_T50-RS-C, the most challenging split of ScanObjectNN. Each clean point cloud is represented by 1024 points. SONN denotes ScanObjectNN.

Method	ModelNet-C	Omni3D	O-LVIS	#Params
Method	(40)	(216)	(1156)	(M)
ULIP-2	1,556	1,558	1,556	85.7
+ Point-Cache	1,556	1,558	1,566	85.7
+ BayesMM	1,560	1,560	1,562	85.7
OpenShape	7,056	7,058	7,116	2,571.9
+ Point-Cache	7,058	7,062	7,150	2,571.9

Appendix

Table J : Memory usage (MB) comparison across different datasets. Numbers below each dataset name indicate the number of classes. Point-Cache denotes the hierarchical variant.

Method	ULIP	ULIP2	OpenShape	Uni3D
Vanilla	11.25	11.25	8.60	7.72
+ Point-Cache [ 24 ]	11.17	11.17	8.57	7.62
+ BayesMM	10.90	10.91	8.28	7.41

Appendix

Table K: Inference throughput (samples per second) on S-OBJ_ONLY. All experiments are conducted with a batch size of 1 on an RTX 3090 GPU. Point-Cache refers to the hierarchical variant of Point-Cache here.

Bayesian Model Averaging

See all 8 papers

MAMVI: 3D Test-Time Adaptation via Masked Multi-View Point Clouds
Jun 11, 2026Inseok Kong, Geunyoung Jung, Jiyoung JungTest-Time Adaptation Point Cloud Classification
Department of Geo Informatics, University of Seoul · Department of Artificial Intelligence, University of Seoul
Domain Generalizable Adaptation of 3D Vision-Language Models via Regularized Fine-Tuning
Jun 16, 2026Sneha Paul, Zachary Patterson, Nizar BouguilaMultimodal Robustness VLM Adaptation
Concordia University, Canada
P3T: Prototypical Point-level Prompt Tuning with Enhanced Generalization for 3D Vision-Language Models
Apr 17, 2026Geunyoung Jung, Soohong Kim, Kyungwoo Song +1Point Cloud Classification VLM Adaptation
Department of Artificial Intelligence, University of Seoul, South Korea · Department of Applied Statistics, Yonsei University, South Korea

Jun 11, 2026J/K move · Enter open · S save

MAMVI: 3D Test-Time Adaptation via Masked Multi-View Point Clouds

Inseok Kong, Geunyoung Jung, Jiyoung Jung

Department of Geo Informatics, University of Seoul · Department of Artificial Intelligence, University of Seoul

Test-Time Adaptation Point Cloud Classification Point Cloud Learning Multi-View Learning

Open paper AskPDF arXiv

3D point cloud models suffer significant performance degradation under distribution shifts caused by sensor noise, occlusions, and environmental changes. Test-time adaptation (TTA) has emerged as a practical paradigm for mitigating this issue during inference. Recently, leveraging multi-view augmentation has shown promise in improving 3D TTA performance. However, existing multi-view approaches are often constrained by sequential optimization that treats each view independently. This sequential optimization leads to substantial inference latency due to repetitive optimization steps, making real-time adaptation impractical. To address this, we propose Masked Multi-View Test-Time Adaptation (MAMVI), which replaces sequential optimization with a unified single-step adaptation. Specifically, MAMVI utilizes a hybrid masking strategy that combines fixed ratios for stability with Beta-distributed sampling for diversity. By aggregating losses across multiple views, MAMVI performs adaptation through a single backward pass based on multi-view consensus. Additionally, a confidence-based adaptive learning rate is used to dynamically adjust the adaptation intensity for each sample. Extensive experiments on ModelNet-40C, ShapeNet-C, and ScanObjectNN-C demonstrate that MAMVI achieves state-of-the-art accuracy on ShapeNet-C and ScanObjectNN-C. Moreover, it remains competitive on ModelNet-40C while delivering 4.9-8.9 times faster inference, making it highly suitable for real-time applications. Our code is available at https://github.com/Inseok-kong/MAMVI

Abstract

Bayesian Model Averaging

Explore similar work

MAMVI: 3D Test-Time Adaptation via Masked Multi-View Point Clouds

Domain Generalizable Adaptation of 3D Vision-Language Models via Regularized Fine-Tuning

P3T: Prototypical Point-level Prompt Tuning with Enhanced Generalization for 3D Vision-Language Models