Joint Optimization of Storage and Loading for High-Performance 3D Point Cloud Data Processing

Ke Wang; Yanfei Cao; Xiangzhi Tao; Naijie Gu; Jun Yu; Zhengdong Wang; Shouyang Dong; Fan Yu

cs.CVMar 16, 2026

Joint Optimization of Storage and Loading for High-Performance 3D Point Cloud Data Processing

Authors: Ke Wang, Yanfei Cao, Xiangzhi Tao, Naijie Gu, Jun Yu, Zhengdong Wang, Shouyang Dong, Fan Yu,

Abstract

With the rapid development of computer vision and deep learning, significant advancements have been made in 3D vision, partic- ularly in autonomous driving, robotic perception, and augmented reality. 3D point cloud data, as a crucial representation of 3D information, has gained widespread attention. However, the vast scale and complexity of point cloud data present significant chal- lenges for loading and processing and traditional algorithms struggle to handle large-scale datasets.The diversity of storage formats for point cloud datasets (e.g., PLY, XYZ, BIN) adds complexity to data handling and results in inefficiencies in data preparation. Al- though binary formats like BIN and NPY have been used to speed up data access, they still do not fully address the time-consuming data loading and processing phase. To overcome these challenges, we propose the .PcRecord format, a unified data storage solution designed to reduce the storage occupation and accelerate the processing of point cloud data. We also introduce a high-performance data processing pipeline equipped with multiple modules. By leveraging a multi-stage parallel pipeline architecture, our system optimizes the use of computational resources, significantly improving processing speed and efficiency. This paper details the im- plementation of this system and demonstrates its effectiveness in addressing the challenges of handling large-scale point cloud datasets.On average, our system achieves performance improvements of 6.61x (ModelNet40), 2.69x (S3DIS), 2.23x (ShapeNet), 3.09x (Kitti), 8.07x (SUN RGB-D), and 5.67x (ScanNet) with GPU and 6.9x, 1.88x, 1.29x, 2.28x, 25.4x, and 19.3x with Ascend.

Figures & tables

Figure 1: Overview of the proposed point cloud data processing pipeline with (a) Format Standardization Module, (b) Distributed Processing Module, (c)Data Loading and Processing Pipeline, and (d) Autotune Module.

Figure 2: Workflow of the format standardization module.

Figure 4: The schema of ModelNet40 dataset.

Figure 5: Workflow of OBS streaming dataset loading system.

Figure 6: Workflow of data parallel processing and training on multiple devices.

Figure 7: Diagram of order-preserving strategy of data processing pipeline.

Figure 8: Workflow of pipeline update process.

Dataset	workers	map operator	batch size
ModelNet40	8	normalize, translate	32
S3DIS	8	normalize, jitter, rotate, translate	4
ShapeNet	8	random scale, translate	10
Kitti	8	normalize	32
Sunrgbd	8	Augment RGB color, random scale, rotate, flip along the YZ plane	4
ScanNet	8	Random Cropping, Downsampling	32

Table 1: The processing configurations of different point cloud datasets.

Dataset	ModelNet40			S3DIS			ShapeNet			Kitti			Sunrgbd			ScanNet
Metric	Cost Time (s), Average Memory Usage (%), Average CPU Usage (%)
MindSpore	17.24	0.45%	93.37%	4	1.24%	128.10%	68.15	0.55%	94.71%	8.94	0.45%	103.51%	25.85	2.69%	48.18%	6.8	0.72%	97.81%
PyTorch	39.83	0.37%	21.55%	6.26	1.13 %	99.38%	7.99	0.32%	66.40%	9.18	0.32 %	54.44%	18.57	2.84%	61.50%	7.06	0.59%	90.03%
Tensorflow	76.05	1.57%	98.72%	16.29	1.74%	106.81%	59.27	1.54%	103.47%	19.4	0.49%	99.60%	29.27	3.93%	91.43%	16.96	1.00%	70.92%
Keras	85.29	0.39%	49.10%	18.34	1.06%	54.76%	63.69	0.38%	51.73%	35.06	0.51%	52.78%	41.97	2.47%	34.87%	36.71	0.70%	45.86%
Ours	2.61	0.24 %	114.26 %	2.33	1.24%	128.41 %	3.58	0.27 %	121.47 %	2.89	0.43%	169.15 %	2.3	0.27 %	115.90 %	1.2	0.26 %	108.12 %

Table 2: Performance comparison of various frameworks on different datasets with GPU.The three values for each dataset are Cost Time (s), Average Memory Usage (%), and Average CPU Usage (%).The best values are shown in bold.

Dataset	ModelNet40			S3DIS			ShapeNet			Kitti			Sunrgbd			ScanNet
Metric	Cost Time (s), Average Memory Usage (%), Average CPU Usage (%)
MindSpore	12.8	0.11%	103.03%	14.87	0.43%	105.67%	10.37	0.10%	127.37%	3.91	0.18%	109.50%	21.49	0.88%	120.84%	3.12	0.22%	99.21%
PyTorch	39.62	0.12%	36.77%	26.65	0.45%	95.26%	5.1	0.09 %	118.98%	11.1	0.12 %	36.42%	17.78	0.88%	98.78%	2.51	0.20%	99.62%
Tensorflow	77.76	0.14%	107.27%	34.46	0.47%	117.58%	35.65	0.17%	115.57%	24.26	0.20%	104.94%	24.46	0.93%	120.72%	11.17	0.26%	109.10%
Keras	77.94	0.12%	106.92%	36.65	0.33%	111.53%	35.34	0.12%	113.82%	23.51	0.20%	105.40%	25.05	0.86%	118.05%	12.9	0.26%	108.86%
Ours	1.84	0.08 %	161.00 %	7.92	0.12 %	283.55 %	3.95	0.09 %	150.58 %	1.71	0.19%	299.73 %	0.7	0.08 %	164.66 %	0.13	0.07 %	189.77 %

Table 3: Performance comparison of various frameworks on different datasets with Ascend.The three values for each dataset are Cost Time (s), Average Memory Usage (%), and Average CPU Usage (%).The best values are shown in bold.

Experiment	Categories
Hardware Combination	NPU+CPU GPU+CPU
Framework Version	TensorFlow 2.13.1,PyTorch 2.4.0,MindSpore 2.3.1,Keras 2.13.1

Table 4: Experiment Setting

Figure 9: Comparison of storage space before and after the format conversion.

Figure 10: The deviation rate of time spent on data loading and processing with different datasets for 20 epochs.

Figure 11: Normalized time consumption of loading and processing different datasets on GPU and Ascend.

Figure 12: Normalized average CPU usage of loading and processing different datasets on GPU and Ascend.

Figure 13: Normalized time consumption of reading and processing different datasets with the distributed setting.

Dataset	MindSpore		Ours
Dataset	single	mul	single	mul
ModelNet40	17.24	22.07	2.61	0.58
S3DIS	4.00	5.94	2.33	2.75
ShapeNet	68.15	13.57	3.58	1.17
Kitti	8.94	6.73	2.89	1.10
Sunrgbd	25.85	6.36	2.30	0.35

Table 5: Performance comparison between MindSpore and our method with single and multiple devices.

Figure 14: The real-time throughput(items/sec) when processing each batch of point cloud dataset.The right vertical axis is for our method if it exists.

Figure 15: MindSpore vs. Ours in terms of CPU usage with Dataset Kitti.

Figure 16: MindSpore’s memory usage curve for 10 epochs with Dataset Kitti.

Figure 17: MindSpore vs. Ours in terms of memory usage with Dataset Kitti.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 18: MindSpore vs. Ours in terms of memory usage with Dataset Kitti.

Figure 19: MindSpore vs. Ours in terms of memory usage with Dataset Kitti.

Figure 20: MindSpore vs. Ours in terms of memory usage with Dataset Kitti.

Figure 21: MindSpore vs. Ours in terms of memory usage with Dataset Kitti.

High-Performance Computing

High-performance computing (HPC) uses parallel systems and optimized software to solve computationally demanding scientific and engineering problems.

See all 143 papers

Apr 27, 2026

PointTransformerX: Portable and Efficient 3D Point Cloud Processing without Sparse Algorithms

3D point cloud perception remains tightly coupled to custom CUDA operators for spatial operations, limiting portability and efficiency on non-NVIDIA, AMD, and embedded hardware. We introduce PointTransformerX (PTX), a fully PyTorch-native vision transformer backbone for 3D point clouds, removing all custom CUDA operators and external libraries while retaining competitive accuracy. PTX introduces 3D-GS-RoPE, a rotary positional embedding that encodes 3D spatial relationships directly in self-attention without neighborhood construction, and further replaces sparse convolutional patch embedding with a linear projection. PTX explores inference-time scaling of attention windows to improve accuracy without retraining. With a redesigned feed-forward network, PTX achieves 98.7% of PointTransformer V3's accuracy on ScanNet with 79.2% fewer parameters and executing 1.6\times faster while requiring just 253 MB memory. PTX runs natively on NVIDIA GPUs, AMD GPUs (ROCm), and CPUs, providing an efficient and portable foundation for point cloud perception.

Laurenz Reichardt, Nikolas Ebert, Oliver Wasenmüller

Mannheim University of Applied Sciences, Germany

Rotary Positional Embeddings 3D ViTs Efficient ViTs Point Cloud Understanding

Ask PDF

Dec 15, 2025

LitePT: Lighter Yet Stronger Point Transformer

Modern neural architectures for 3D point cloud processing contain both convolutional layers and attention blocks, but the best way to assemble them remains unclear. We analyse the role of different computational blocks in 3D point cloud networks and find an intuitive behaviour: convolution is adequate to extract low-level geometry at high-resolution in early layers, where attention is expensive without bringing any benefits; attention captures high-level semantics and context in low-resolution, deep layers more efficiently, where convolution inflates the parameter count. Guided by this design principle, we propose a new, improved 3D point cloud backbone that employs convolutions in early stages and switches to attention for deeper layers. To avoid the loss of spatial layout information when discarding redundant convolution layers, we introduce a novel, parameter-free 3D positional encoding, PointROPE. The resulting LitePT model has

3.6\times

fewer parameters, runs

2\times

faster, and uses

2\times

less memory than the state-of-the-art Point Transformer V3, but nonetheless matches or outperforms it on a range of tasks and datasets. Code and models are available at: https://github.com/prs-eth/LitePT.

Yuanwen Yue, Damien Robert, Jianyuan Wang +4

ETH Zurich · University of Oxford · University of Zurich

Point Cloud Learning Point Transformer

Ask PDF

May 16, 2026

A Survey on Deep Learning Architectures for Point Cloud Classification and Segmentation

Point cloud stands as the most widely adopted format for representing 3D shapes and scenes due to its simplicity and geometric fidelity. However, its inherent unordered and irregular nature, exacerbated by sensor noise and occlusions, introduces unique challenges for machine learning based methodologies. To combat these issues, diverse strategies have been developed, including converting to a format that has orderliness, extracting local geometry, and permutation-invariant or self-attention-based processing. In this paper, our focus is directed towards deep learning models for three fundamental tasks in 3D vision: point cloud classification, part segmentation, and semantic segmentation. We begin by formally defining point cloud data, followed by an in-depth discussion on its structural characteristics. Then, we categorize notable works based on their backbone structure and evaluate their performance on popular benchmarks. Beyond empirical comparison, we offer insights into architectural innovations and limitations. We also outline open challenges and promising future directions for 3D point cloud understanding.

Minhas Kamal, Hiranya Garbha Kumar, Balakrishnan Prabhakaran

State University of New York at Albany, USA

3D Point Cloud Segmentation 3D Part Segmentation Point Cloud Classification Point Cloud Learning 3D Semantic Segmentation

Ask PDF

Abstract

High-Performance Computing

Explore similar work

PointTransformerX: Portable and Efficient 3D Point Cloud Processing without Sparse Algorithms

LitePT: Lighter Yet Stronger Point Transformer

A Survey on Deep Learning Architectures for Point Cloud Classification and Segmentation