Efficient Point Cloud Processing with High-Dimensional Positional Encoding and Non-Local MLPs

Yanmei Zou; Hongshan Yu; Yaonan Wang; Zhengeng Yang; Xieyuanli Chen; Kailun Yang; Naveed Akhtar

cs.CVMar 4, 2026

Efficient Point Cloud Processing with High-Dimensional Positional Encoding and Non-Local MLPs

Authors: Yanmei Zou, Hongshan Yu, Yaonan Wang, Zhengeng Yang, Xieyuanli Chen, Kailun Yang, Naveed Akhtar

Organizations: the School of Artificial Intelligence and Robotics, Quanzhou Institute of Industrial Design and Machine Intelligence Innovation, Hunan University, Changsha 410082, China. · the College of Engineering and Design, Hunan Normal University, Changsha 410081, China. · the College of Intelligence Science and Technology, National University of Defense Technology, Changsha 410073, China. · the School of Computing and Information Systems, The University of Melbourne, 3052 Victoria, Australia.

Abstract

Multi-Layer Perceptron (MLP) models are the foundation of contemporary point cloud processing. However, their complex network architectures obscure the source of their strength and limit the application of these models. In this article, we develop a two-stage abstraction and refinement (ABS-REF) view for modular feature extraction in point cloud processing. This view elucidates that whereas the early models focused on ABS stages, the more recent techniques devise sophisticated REF stages to attain performance advantages. Then, we propose a High-dimensional Positional Encoding (HPE) module to explicitly utilize intrinsic positional information, extending the ``positional encoding'' concept from Transformer literature. HPE can be readily deployed in MLP-based architectures and is compatible with transformer-based methods. Within our ABS-REF view, we rethink local aggregation in MLP-based methods and propose replacing time-consuming local MLP operations, which are used to capture local relationships among neighbors. Instead, we use non-local MLPs for efficient non-local information updates, combined with the proposed HPE for effective local information representation. We leverage our modules to develop HPENets, a suite of MLP networks that follow the ABS-REF paradigm, incorporating a scalable HPE-based REF stage. Extensive experiments on seven public datasets across four different tasks show that HPENets deliver a strong balance between efficiency and effectiveness. Notably, HPENet surpasses PointNeXt, a strong MLP-based counterpart, by 1.1% mAcc, 4.0% mIoU, 1.8% mIoU, and 0.2% Cls. mIoU, with only 50.0%, 21.5%, 23.1%, 44.4% of FLOPs on ScanObjectNN, S3DIS, ScanNet, and ShapeNetPart, respectively. Source code is available at https://github.com/zouyanmei/HPENet_v2.git.

Figures & tables

Fig. 1: Segmentation performance of HPENet V2 on S3DIS [ 4 ] . Bubble diameter is proportional to model parameter and the legend lists reference sizes of 1M, 3M, 20M, and 40M. HPENet V2 achieves higher throughput and competitive or better mIoU than state-of-the-art methods, including PointMetaBase [ 5 ] , PointNeXt [ 6 ] , and PointVector [ 7 ] , while using fewer parameters. Compared with our previous HPENet [ 8 ] , HPENet V2 delivers comparable performance while using fewer parameters and achieving about $2.2\times$ faster inference.

Fig. 2: HPENet V2 architecture for semantic segmentation. The network delineates between the Abstraction (ABS) and Refinement (REF) stages of feature extraction and employs the proposed High-dimensional Positional Encoding (HPE) module in both stages. The differences between HPENet V2 and HPENet [ 8 ] are highlighted with red borders, such as the Backward Fusion Module (BFM) and the non-local MLPs.

Fig. 3: Illustration of abstraction and refinement (ABS-REF) perspective. Left: The proposed ABS-REF view of point cloud models is analogous to subsampling and convolution block view in image models. The shown ABS-REF column expands the abstraction and refinement stages. Right: Representative instantiations of the ABS-REF framework. Whereas early methods, e.g., PointNet++ [ 13 ] , PointConv [ 14 ] , ignore the REF stage, more recent techniques, e.g., Point Transformer [ 11 ] and PointMixer [ 19 ] , achieve higher performance by accounting for the REF stage in point cloud models. Abbreviations include SOP: Symmetric OPeration, OP: aggregation OPeration, PT: Point Transformer, and HPE: proposed High-dimensional Positional Encoding.

Fig. 4: Illustration of separable Abstraction and Refinement (ABS-REF) perspective. Left: The separable ABS-REF view also belongs to the proposed ABS-REF by disentangling the intra-set operation and inter-set operation into non-local MLPs and the reduction operation. Right: Representative instantiations of the separable ABS-REF framework and the evolution of HPENets, i.e., from HPENet to HPENet V2. Please refer to Fig. 3 for the feature propagation in the proposed ABS-REF view.

Fig. 5: Illustration of local aggregation in the ABS stages of MLP-based methods. (a) PreConv processes input point sets with non-local MLPs before sampling operations. (b) Traditional Conv processes neighboring point sets generated by grouping operations with local MLPs. (c) ProConv processes sampled point sets with non-local aggregation after reduction operations. Please refer to the text for further notations.

Line	ABS	REF	OA (%)	mAcc (%)	mIoU (%)	$\bigtriangleup$	Param. (M)	FLOPs (G)	TP (ins./sec.)
1	Conv	Conv	89.0 $\pm$ 0.1	73.0 $\pm$ 0.4	66.6 $\pm$ 0.4	-	1.58	6.93	317
2	Conv*	Conv*	86.3 $\pm$ 0.3	67.4 $\pm$ 0.6	60.5 $\pm$ 0.6	-6.1	1.58	6.76	337
3	PreConv	Conv	88.3 $\pm$ 0.4	71.9 $\pm$ 0.3	65.6 $\pm$ 0.4	-1.0	1.58	5.09	355
4	ProConv	Conv	88.1 $\pm$ 0.1	71.7 $\pm$ 0.5	65.1 $\pm$ 0.4	-1.5	1.58	4.90	380
5	Conv	PreConv	88.6 $\pm$ 0.3	72.1 $\pm$ 0.4	65.8 $\pm$ 0.6	-0.8	1.58	3.01	388
6	Conv	ProConv	87.9 $\pm$ 0.1	70.4 $\pm$ 0.4	63.7 $\pm$ 0.4	-2.9	1.58	3.01	388

TABLE I: A study of local aggregation on S3DIS Area-5.

\rm\mathbf{Conv}

\rm\mathbf{PreConv}

, and

\rm\mathbf{ProConv}

represent different local aggregation,

\rm\mathbf{PE}

refers to Positional Encoding, and

\rm\mathbf{HPE}

denotes the proposed High-dimensional Positional Encoding.

\mathbf{\Delta}

represents the difference in mIoU between the current row and the first row.

\rm\mathbf{PreConv^{\dagger}}

indicates the use of Conv in the first ABS stage. Please refer to Sec. V and Sec. V-B for the evaluation metrics.

Methods	OA	mAcc	Param.	FLOPs	TP
PointNet [ 12 ]	68.2	63.4	3.5	0.9	6846
PointCNN [ 72 ]	78.5	75.1	0.6	-	-
PointNet++ [ 13 ]	77.9	75.4	1.5	1.7	3019
DGCNN [ 60 ]	78.1	73.6	1.8	4.8	1093
PRANet [ 73 ]	82.1	79.1	2.3	-	-
PointMLP [ 20 ]	85.7	84.4	13.2	31.4	370

TABLE II: 3D object classification on ScanObjectNN [ 25 ] . The best results are highlighted in bold , while the second-best results are underlined . Results marked with an asterisk (*) indicate released values.

Methods	OA	mAcc	Param.	FLOPs	TP
PointNet [ 12 ]	89.2	86.2	3.5	0.9	6848
PointNet++ [ 13 ]	91.9	-	1.5	1.7	3019
PointCNN [ 72 ]	92.2	88.1	0.6	-	-
PointConv [ 14 ]	92.5	-	18.6	-	-
DGCNN [ 60 ]	92.9	90.2	1.8	4.8	1093
DeepGCN [ 74 ]	93.6	90.9	2.2	3.9	629

TABLE III: 3D object classification on ModelNet40 [ 26 ] .

Methods	S3DIS Area-5			S3DIS 6-fold			Param.	FLOPs	TP
Methods	mIoU	OA	mAcc	mIoU	OA	mAcc	M	G	ins./sec.
PointNet++ [ 13 ]	53.5	83.0	-	54.5	81.0	67.1	1.0	7.2	332
PointCNN [ 72 ]	57.3	85.9	63.9	65.4	88.1	75.6	0.6	-	-
KPConv [ 47 ]	67.1	-	72.8	70.6	-	79.1	15.0	-	-
RepSurf-U [ 75 ]	68.9	90.2	76.0	74.3	89.9	82.6	1.0	1.0	-
ASSANet [ 23 ]	68.0	89.7	74.3	-	-	-	115.6	36.2	-

TABLE IV: Scene semantic segmentation on S3DIS [ 4 ] .

Fig. 6: Representative visual comparison of semantic segmentation results on S3DIS Area-5. “ST” denotes Stratified Transformer.

Methods	mIoU	Param.	FLOPs	TP
PointNet++ [ 13 ]	53.5	1.0	7.2	332
RepSurf-U [ 75 ]	70.0	1.0	1.0	-
Point Transformer [ 11 ]	70.6	7.8	5.6	-
MinkowskiNet [ 9 ]	72.2	37.9	4.7	-
Fast Point Transformer [ 54 ]	72.5	37.9	-	-
Stratified Transformer [ 24 ]	74.3	18.8	16.1	-

TABLE V: Scene semantic segmentation on ScanNet V2 [ 27 ] val. set.

Methods	mIoU	Param.	FLOPs
SPVNAS [ 79 ]	64.7	10.8	64.5
Cylinder3D [ 80 ]	64.3	55.8	134.2
WaffleIron [ 81 ]	68.0	-	-
SphereFormer [ 78 ]	67.8	32.3	89.8
RangeFormer [ 82 ]	67.6	24.3	-
MinkowskiNet [ 9 ]	63.8	37.9	-

TABLE VI: Scene semantic segmentation on SemanticKITTI [ 28 ] val. set.

Methods	Cls. mIoU	Ins. mIoU	Param.	FLOPs	TP
PointNet [ 12 ]	80.4	83.7	3.6	4.9	-
PointNet++ [ 13 ]	81.9	85.1	1.1	5.5	-
DGCNN [ 60 ]	82.3	85.2	1.3	12.4	-
KPConv [ 47 ]	85.1	86.4	-	-	-
PointMLP [ 20 ]	84.6	86.1	-	-	-
DTNet [ 83 ]	-	86.2	-	-	-

TABLE VII: 3D object part segmentation on ShapeNetPart [ 29 ] .

Fig. 7: Representative qualitative comparison of semantic segmentation on ScanNet. Visualizations for Fast Point Transformer (FPT) exclude points annotated with the ignore label.

Model	[email protected]	bed	table	sofa	chair	toilet	desk	dresser	nightstand	shelf	bathtub
VoteNet [ 70 ]	57.7	83.0	47.3	64.0	75.3	90.1	22.0	29.8	62.2	28.8	74.4
VoteNet ‡ [ 70 ]	59.7	84.8	49.6	67.8	77.6	87.4	24.3	29.3	61.9	32.1	82.1
BRNet [ 85 ]	61.1	86.9	51.8	66.4	77.4	91.3	29.6	35.9	65.9	29.7	76.2
3DETR [ 86 ]	59.1	84.6	52.6	65.3	72.4	91.0	34.3	29.6	61.4	28.5	69.8
VoteNet ‡ + $HPE_{MLP}$	61.2	85.0	50.5	67.7	78.5	91.1	28.5	35.0	64.1	34.3	77.1

TABLE VIII: 3D object detection results on SUN RGB-D validation set with [email protected]. ‡ donates that the model is implemented on MMDetection3D [ 84 ] .

Networks( $C_{e}$ , $B$ )	HPE	OA	mAcc	mIoU	$\triangle$	Param.	FLOPs	TP
HPENet_dv (32, [0, 0, 0, 0])		82.7 $\pm$ 0.6	58.3 $\pm$ 0.7	51.5 $\pm$ 0.2	-	0.5	0.8	509
HPENet_dv (32, [1, 1, 1, 1])		85.8 $\pm$ 0.5	65.5 $\pm$ 0.3	58.8 $\pm$ 0.6	+7.3	1.6	1.2	452
HPENet_dv (32, [1, 1, 1, 1])	PE	89.3 $\pm$ 0.3	73.4 $\pm$ 0.3	66.7 $\pm$ 0.4	+15.2	1.6	1.4	328
HPENet_dv (32, [1, 1, 1, 1])	HPE	90.0 $\pm$ 0.1	74.5 $\pm$ 0.8	68.1 $\pm$ 0.4	+16.6	1.8	3.2	309
HPENet_dv ⋆		83.5 $\pm$ 0.5	60.1 $\pm$ 0.8	53.2 $\pm$ 0.7	-5.6	1.6	1.2	507

TABLE IX: Ablation study on S3DIS Area-5 demonstrating the efficacy of ABS-REF view, and contribution of HPE modules.

\mathbf{\Delta}

represents the difference in mIoU between the current row and the first row.

Fig. 8: Qualitative comparisons among PointMetaBase [ 5 ] (first row), HPENet (second row), and Ground Truth (third row) on ShapeNetPart [ 29 ] for 3D object part segmentation.

Method	[email protected]	[email protected]
VoteNet [ 70 ]	58.6	33.5
VoteNet ‡ [ 70 ]	63.8	44.2
MLCVNet [ 89 ]	64.5	41.4
3DETR [ 86 ]	65.0	47.0
GroupFree3D ‡ [ 87 ]	68.2	52.6
GroupFree3D + X-3D [ 45 ]	69.0	51.1

TABLE X: 3D object detection on ScanNet V2 validation set. ‡ donates that the model is implemented on MMDetection3D [ 84 ] .

Line	Type	ABS	REF	BFM	OA	mAcc	mIoU	$\triangle$	Param.(M)	FLOPs(G)	TP(ins./sec.)
1					87.5 $\pm$ 0.4	70.4 $\pm$ 0.5	63.9 $\pm$ 0.3	-	15.4	10.4	176
2	HPE	$\surd$			90.2 $\pm$ 0.1	76.1 $\pm$ 0.4	70.0 $\pm$ 0.4	+6.1	15.8	13.7	160
3	HPE		$\surd$		91.0 $\pm$ 0.1	76.9 $\pm$ 0.6	70.7 $\pm$ 0.5	+6.8	15.8	14.5	148
4	HPE	$\surd$	$\surd$		91.5 $\pm$ 0.2	77.9 $\pm$ 0.3	72.2 $\pm$ 0.2	+8.3	16.0	17.7	137
5	HPE	$\surd$	$\surd$	$\surd$	91.5 $\pm$ 0.1	78.4 $\pm$ 0.5	72.3 $\pm$ 0.2	+8.4	16.1	18.2	128
6	PE	$\surd$	$\surd$		90.9 $\pm$ 0.2	77.0 $\pm$ 0.5	70.9 $\pm$ 0.5	-1.3	15.4	10.8	144

TABLE XI: Ablation study for positional encoding and backward fusion module with HPENet V2-XL on S3DIS Area-5 justifying HPE use in both ABS and REF stages.

\mathbf{\Delta}

in lines 1 through 5 indicates the difference in mIoU between the corresponding row and the first row, while

\mathbf{\Delta}

in lines 6 through 8 represents the difference in mIoU between the corresponding row and the fourth row.

Dimension	OA	mAcc	mIoU	$\triangle$	Param.	FLOPs	TP
3	90.9 $\pm$ 0.2	77.0 $\pm$ 0.5	70.9 $\pm$ 0.5	-	15.3	10.8	144
C/8	91.6 $\pm$ 0.2	77.9 $\pm$ 0.2	71.9 $\pm$ 0.3	+1.0	15.6	14.1	141
C/4	91.5 $\pm$ 0.2	77.9 $\pm$ 0.3	72.2 $\pm$ 0.2	+1.3	16.0	17.7	137
C/2	91.4 $\pm$ 0.1	77.4 $\pm$ 0.2	71.5 $\pm$ 0.1	+0.6	16.7	24.9	129
C	91.4 $\pm$ 0.3	78.2 $\pm$ 0.7	71.8 $\pm$ 0.4	+0.9	18.1	39.3	116

TABLE XII: Ablation study on dimension of

HPE_{MLP}

with HPENet V2-XL on S3DIS Area-5. ‘C’ denotes the feature channel number.

\mathbf{\Delta}

represents the difference in mIoU between the current row and the first row.

Methods	ABS (MLPs)	REF (PT layers)	BFM	OA	mACC	mIoU	$\triangle$	Param.(M)	FLOPs(G)
Point Transformer [ 11 ]	Local MLPs	PE		90.4	74.8	68.2	-	7.8	5.6
+Non-local MLPs	Non-local MLPs	PE		90.7	76.3	69.7	+1.5	7.8	4.9
+HPE	Non-local MLPs + HPE	HPE		90.5	76.4	70.3	+2.1	7.8	5.2
+BFM	Non-local MLPs + HPE	HPE	$\surd$	90.9	76.6	70.7	+2.5	7.9	5.3
Stratified Transformer [ 24 ]	Local MLPs	cRPE		91.7	77.4	71.4	-	8.0	12.3
+Non-local MLPs	Non-local MLPs	cRPE		90.9	78.9	72.0	+0.6	8.0	11.1

TABLE XIII: Ablation of the proposed modules on Point Transformer (PT) [ 11 ] and Stratified Transformer (ST) [ 24 ] for S3DIS Area-5.

\Delta

denotes the mIoU difference relative to the baseline row. cRPE (Contextual Relative Position Encoding) is a spatial relative positional encoding used in ST.

Fig. 9: t-SNE comparison of positional encoding variants for HPENet-dv. Left to right: (a) Baseline, (b) HPE (abs), (c) HPE (mal), (d) HPE.

Fig. 10: Visual comparison of feature-to-label similarity matrices for HPENet V2 variants. The baseline and label matrix are shown for reference.

Fig. 11: Robustness to point density during training and testing on ScanObjectNN across PointNeXt, PointMetaBase, and HPENet V2.

Fig. 12: 3D object classification architecture of HPENet V2. The classification architecture has the same encoder structure as the semantic segmentation model (Fig. 2 in the manuscript).

Fig. 13: More recent representative instantiations of the ABS-REF framework. Abbreviations include SOP: Symmetric OPeration, OP: aggregation OPeration, Two SST blocks: Two Successive Stratified Transformer Blocks.

Methods	Area	OA	mACC	mIoU	Ceiling	Floor	Wall	Beam	Column	Window	Door	Chair	Table	Bookcase	Sofa	Board	Clutter
HPENet V2-S	1	90.8	85.9	77.4	96.3	96.0	84.5	64.9	60.9	81.4	88.0	73.2	83.0	74.5	61.7	71.8	70.3
	2	81.9	66.2	51.9	90.4	86.9	80.1	9.0	44.5	17.5	63.4	49.6	43.5	69.3	49.7	30.0	41.1
	3	91.4	85.2	76.5	96.3	98.2	84.4	66.8	60.5	48.8	87.7	72.2	81.5	74.0	68.5	82.1	73.4
	4	86.9	71.7	60.9	92.0	97.6	82.5	9.0	45.7	36.5	68.2	66.3	76.1	51.6	56.2	52.7	57.9
	5	89.4	72.4	65.6	93.6	98.5	83.4	0.0	32.3	52.7	68.0	81.7	88.8	57.9	72.4	66.2	57.4
	6	93.5	90.7	83.6	96.7	97.9	88.2	77.7	77.2	86.3	91.2	79.7	88.2	72.4	76.2	79.3	75.8

TABLE XIV: Detailed semantic segmentation results (%) on S3DIS [ 4 ] with 6-fold cross-validation.

Method	mIoU	car	bicycle	motorcycle	truck	other-vehicle	person	bicyclist	motorcyclist	road	parking	sidewalk	other-ground	building	fence	vegetation	trunk	terrain	pole	traffic-sign
HPENet V2-S	66.9	95.5	49.9	79.3	85.7	52.2	75.3	90.8	3.6	93.9	49.9	82.8	0.7	90.0	62.8	90.0	72.7	79.4	65.5	50.4
HPENet V2-B	67.0	95.7	43.8	77.3	91.7	55.0	75.6	91.5	2.1	94.0	49.0	82.7	6.8	90.6	66.0	88.5	70.4	75.4	65.4	51.6
HPENet V2-L	67.4	96.2	52.7	77.1	88.1	64.2	73.6	88.3	0.1	94.0	54.9	82.0	0.4	91.1	67.6	88.6	70.3	75.8	64.8	51.4
HPENet V2-XL	69.3	96.5	55.3	78.8	92.8	66.2	78.4	86.1	25.2	94.6	48.6	83.9	0.3	91.7	69.2	87.7	69.8	74.0	65.4	51.7

TABLE XV: Detailed semantic segmentation results (%) on SemanticKITTI.

Fig. 14: Representative qualitative results of HPENet V2 and the balanced MLP-based method PointMetaBase [ 5 ] on S3DIS Area-5. The main differences are highlighted in red frames.

Fig. 15: Representative qualitative results of HPENet V2 and the balanced MLP-based method PointMetaBase [ 5 ] on S3DIS Area-5. The main differences are highlighted in red frames.

Fig. 16: Representative qualitative results of HPENet V2 and the balanced MLP-based method PointMetaBase [ 5 ] on ScanNet V2. The main differences are highlighted in red frames.

Point Cloud Learning

Point Cloud Learning develops methods for representing and interpreting unordered 3D point sets from LiDAR, depth sensors, radar, and 3D models.

See all 158 papers

LitePT: Lighter Yet Stronger Point Transformer
Dec 15, 2025Yuanwen Yue, Damien Robert, Jianyuan Wang +4Point Cloud Learning Point Transformer
ETH Zurich · University of Oxford · University of Zurich
SLNet: A Super-Lightweight Geometry-Adaptive Network for 3D Point Cloud Recognition
Mar 8, 2026Mohammad Saeid, Amir Salarpour, Pedram MohajerAnsari +1Point Cloud Learning Point Cloud Classification
Sirjan University of Technology, Sirjan, Iran · Clemson University, Clemson, SC, USA
Point Ladder Tuning: Parameter-Efficient Hierarchical Adaptation for 3D Point Cloud Understanding
Jul 21, 2026Junlin Chang, Longhao Zou, Rui Li3D Point Cloud Segmentation Point Cloud Classification
Beihang University, Beijing · Pengcheng Laboratory, Shenzhen

Dec 15, 2025J/K move · Enter open · S save

LitePT: Lighter Yet Stronger Point Transformer

Yuanwen Yue, Damien Robert, Jianyuan Wang, Sunghwan Hong+3

ETH Zurich · University of Oxford · University of Zurich

Point Cloud Learning Point Transformer

Open paper AskPDF arXiv

Modern neural architectures for 3D point cloud processing contain both convolutional layers and attention blocks, but the best way to assemble them remains unclear. We analyse the role of different computational blocks in 3D point cloud networks and find an intuitive behaviour: convolution is adequate to extract low-level geometry at high-resolution in early layers, where attention is expensive without bringing any benefits; attention captures high-level semantics and context in low-resolution, deep layers more efficiently, where convolution inflates the parameter count. Guided by this design principle, we propose a new, improved 3D point cloud backbone that employs convolutions in early stages and switches to attention for deeper layers. To avoid the loss of spatial layout information when discarding redundant convolution layers, we introduce a novel, parameter-free 3D positional encoding, PointROPE. The resulting LitePT model has

3.6\times

fewer parameters, runs

2\times

faster, and uses

2\times

less memory than the state-of-the-art Point Transformer V3, but nonetheless matches or outperforms it on a range of tasks and datasets. Code and models are available at: https://github.com/prs-eth/LitePT.

Abstract

Point Cloud Learning

Explore similar work

LitePT: Lighter Yet Stronger Point Transformer

SLNet: A Super-Lightweight Geometry-Adaptive Network for 3D Point Cloud Recognition

Point Ladder Tuning: Parameter-Efficient Hierarchical Adaptation for 3D Point Cloud Understanding