Joint Optimization of Storage and Loading for High-Performance 3D Point Cloud Data Processing

Ke Wang; Yanfei Cao; Xiangzhi Tao; Naijie Gu; Jun Yu; Zhengdong Wang; Shouyang Dong; Fan Yu

cs.CVMar 16, 2026

Joint Optimization of Storage and Loading for High-Performance 3D Point Cloud Data Processing

Authors: Ke Wang, Yanfei Cao, Xiangzhi Tao, Naijie Gu, Jun Yu, Zhengdong Wang, Shouyang Dong, Fan Yu,

Abstract

With the rapid development of computer vision and deep learning, significant advancements have been made in 3D vision, partic- ularly in autonomous driving, robotic perception, and augmented reality. 3D point cloud data, as a crucial representation of 3D information, has gained widespread attention. However, the vast scale and complexity of point cloud data present significant chal- lenges for loading and processing and traditional algorithms struggle to handle large-scale datasets.The diversity of storage formats for point cloud datasets (e.g., PLY, XYZ, BIN) adds complexity to data handling and results in inefficiencies in data preparation. Al- though binary formats like BIN and NPY have been used to speed up data access, they still do not fully address the time-consuming data loading and processing phase. To overcome these challenges, we propose the .PcRecord format, a unified data storage solution designed to reduce the storage occupation and accelerate the processing of point cloud data. We also introduce a high-performance data processing pipeline equipped with multiple modules. By leveraging a multi-stage parallel pipeline architecture, our system optimizes the use of computational resources, significantly improving processing speed and efficiency. This paper details the im- plementation of this system and demonstrates its effectiveness in addressing the challenges of handling large-scale point cloud datasets.On average, our system achieves performance improvements of 6.61x (ModelNet40), 2.69x (S3DIS), 2.23x (ShapeNet), 3.09x (Kitti), 8.07x (SUN RGB-D), and 5.67x (ScanNet) with GPU and 6.9x, 1.88x, 1.29x, 2.28x, 25.4x, and 19.3x with Ascend.

Figures & tables

Figure 1: Overview of the proposed point cloud data processing pipeline with (a) Format Standardization Module, (b) Distributed Processing Module, (c)Data Loading and Processing Pipeline, and (d) Autotune Module.

Figure 2: Workflow of the format standardization module.

Figure 4: The schema of ModelNet40 dataset.

Figure 5: Workflow of OBS streaming dataset loading system.

Figure 6: Workflow of data parallel processing and training on multiple devices.

Figure 7: Diagram of order-preserving strategy of data processing pipeline.

Figure 8: Workflow of pipeline update process.

Dataset	workers	map operator	batch size
ModelNet40	8	normalize, translate	32
S3DIS	8	normalize, jitter, rotate, translate	4
ShapeNet	8	random scale, translate	10
Kitti	8	normalize	32
Sunrgbd	8	Augment RGB color, random scale, rotate, flip along the YZ plane	4
ScanNet	8	Random Cropping, Downsampling	32

Table 1: The processing configurations of different point cloud datasets.

Dataset	ModelNet40			S3DIS			ShapeNet			Kitti			Sunrgbd			ScanNet
Metric	Cost Time (s), Average Memory Usage (%), Average CPU Usage (%)
MindSpore	17.24	0.45%	93.37%	4	1.24%	128.10%	68.15	0.55%	94.71%	8.94	0.45%	103.51%	25.85	2.69%	48.18%	6.8	0.72%	97.81%
PyTorch	39.83	0.37%	21.55%	6.26	1.13 %	99.38%	7.99	0.32%	66.40%	9.18	0.32 %	54.44%	18.57	2.84%	61.50%	7.06	0.59%	90.03%
Tensorflow	76.05	1.57%	98.72%	16.29	1.74%	106.81%	59.27	1.54%	103.47%	19.4	0.49%	99.60%	29.27	3.93%	91.43%	16.96	1.00%	70.92%
Keras	85.29	0.39%	49.10%	18.34	1.06%	54.76%	63.69	0.38%	51.73%	35.06	0.51%	52.78%	41.97	2.47%	34.87%	36.71	0.70%	45.86%
Ours	2.61	0.24 %	114.26 %	2.33	1.24%	128.41 %	3.58	0.27 %	121.47 %	2.89	0.43%	169.15 %	2.3	0.27 %	115.90 %	1.2	0.26 %	108.12 %

Table 2: Performance comparison of various frameworks on different datasets with GPU.The three values for each dataset are Cost Time (s), Average Memory Usage (%), and Average CPU Usage (%).The best values are shown in bold.

Dataset	ModelNet40			S3DIS			ShapeNet			Kitti			Sunrgbd			ScanNet
Metric	Cost Time (s), Average Memory Usage (%), Average CPU Usage (%)
MindSpore	12.8	0.11%	103.03%	14.87	0.43%	105.67%	10.37	0.10%	127.37%	3.91	0.18%	109.50%	21.49	0.88%	120.84%	3.12	0.22%	99.21%
PyTorch	39.62	0.12%	36.77%	26.65	0.45%	95.26%	5.1	0.09 %	118.98%	11.1	0.12 %	36.42%	17.78	0.88%	98.78%	2.51	0.20%	99.62%
Tensorflow	77.76	0.14%	107.27%	34.46	0.47%	117.58%	35.65	0.17%	115.57%	24.26	0.20%	104.94%	24.46	0.93%	120.72%	11.17	0.26%	109.10%
Keras	77.94	0.12%	106.92%	36.65	0.33%	111.53%	35.34	0.12%	113.82%	23.51	0.20%	105.40%	25.05	0.86%	118.05%	12.9	0.26%	108.86%
Ours	1.84	0.08 %	161.00 %	7.92	0.12 %	283.55 %	3.95	0.09 %	150.58 %	1.71	0.19%	299.73 %	0.7	0.08 %	164.66 %	0.13	0.07 %	189.77 %

Table 3: Performance comparison of various frameworks on different datasets with Ascend.The three values for each dataset are Cost Time (s), Average Memory Usage (%), and Average CPU Usage (%).The best values are shown in bold.

Experiment	Categories
Hardware Combination	NPU+CPU GPU+CPU
Framework Version	TensorFlow 2.13.1,PyTorch 2.4.0,MindSpore 2.3.1,Keras 2.13.1

Table 4: Experiment Setting

Figure 9: Comparison of storage space before and after the format conversion.

Figure 10: The deviation rate of time spent on data loading and processing with different datasets for 20 epochs.

Figure 11: Normalized time consumption of loading and processing different datasets on GPU and Ascend.

Figure 12: Normalized average CPU usage of loading and processing different datasets on GPU and Ascend.

Figure 13: Normalized time consumption of reading and processing different datasets with the distributed setting.

Dataset	MindSpore		Ours
Dataset	single	mul	single	mul
ModelNet40	17.24	22.07	2.61	0.58
S3DIS	4.00	5.94	2.33	2.75
ShapeNet	68.15	13.57	3.58	1.17
Kitti	8.94	6.73	2.89	1.10
Sunrgbd	25.85	6.36	2.30	0.35

Table 5: Performance comparison between MindSpore and our method with single and multiple devices.

Figure 14: The real-time throughput(items/sec) when processing each batch of point cloud dataset.The right vertical axis is for our method if it exists.

Figure 15: MindSpore vs. Ours in terms of CPU usage with Dataset Kitti.

Figure 16: MindSpore’s memory usage curve for 10 epochs with Dataset Kitti.

Figure 17: MindSpore vs. Ours in terms of memory usage with Dataset Kitti.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 18: MindSpore vs. Ours in terms of memory usage with Dataset Kitti.

Figure 19: MindSpore vs. Ours in terms of memory usage with Dataset Kitti.

Figure 20: MindSpore vs. Ours in terms of memory usage with Dataset Kitti.

Figure 21: MindSpore vs. Ours in terms of memory usage with Dataset Kitti.

High-Performance Computing

High-performance computing (HPC) uses parallel systems and optimized software to solve computationally demanding scientific and engineering problems.

See all 143 papers

PointTransformerX: Portable and Efficient 3D Point Cloud Processing without Sparse Algorithms
Apr 27, 2026Laurenz Reichardt, Nikolas Ebert, Oliver WasenmüllerRotary Positional Embeddings 3D ViTs
Mannheim University of Applied Sciences, Germany
LitePT: Lighter Yet Stronger Point Transformer
Dec 15, 2025Yuanwen Yue, Damien Robert, Jianyuan Wang +4Point Cloud Learning Point Transformer
ETH Zurich · University of Oxford · University of Zurich
A Survey on Deep Learning Architectures for Point Cloud Classification and Segmentation
May 16, 2026Minhas Kamal, Hiranya Garbha Kumar, Balakrishnan Prabhakaran3D Point Cloud Segmentation 3D Part Segmentation
State University of New York at Albany, USA

Apr 27, 2026J/K move · Enter open · S save

PointTransformerX: Portable and Efficient 3D Point Cloud Processing without Sparse Algorithms

Laurenz Reichardt, Nikolas Ebert, Oliver Wasenmüller

Mannheim University of Applied Sciences, Germany

Rotary Positional Embeddings 3D ViTs Efficient ViTs Point Cloud Understanding

Open paper AskPDF arXiv

3D point cloud perception remains tightly coupled to custom CUDA operators for spatial operations, limiting portability and efficiency on non-NVIDIA, AMD, and embedded hardware. We introduce PointTransformerX (PTX), a fully PyTorch-native vision transformer backbone for 3D point clouds, removing all custom CUDA operators and external libraries while retaining competitive accuracy. PTX introduces 3D-GS-RoPE, a rotary positional embedding that encodes 3D spatial relationships directly in self-attention without neighborhood construction, and further replaces sparse convolutional patch embedding with a linear projection. PTX explores inference-time scaling of attention windows to improve accuracy without retraining. With a redesigned feed-forward network, PTX achieves 98.7% of PointTransformer V3's accuracy on ScanNet with 79.2% fewer parameters and executing 1.6\times faster while requiring just 253 MB memory. PTX runs natively on NVIDIA GPUs, AMD GPUs (ROCm), and CPUs, providing an efficient and portable foundation for point cloud perception.

Abstract

High-Performance Computing

Explore similar work

PointTransformerX: Portable and Efficient 3D Point Cloud Processing without Sparse Algorithms

LitePT: Lighter Yet Stronger Point Transformer

A Survey on Deep Learning Architectures for Point Cloud Classification and Segmentation