Position Anchor Tuning: Towards Efficient Adaptation of Pre-Trained Point Cloud Transformers

Zheng Liu; Xin Gao; Jinchao Zhu; Gao Huang

cs.CVSep 16, 2026

Position Anchor Tuning: Towards Efficient Adaptation of Pre-Trained Point Cloud Transformers

Authors: Zheng Liu, Xin Gao, Jinchao Zhu, Gao Huang

Organizations: School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing 100083, China, and also with the Beijing Engineering Research Center of Industrial Spectrum Imaging, Beijing 100083, China · College of Software, Nankai University, Tianjin 300350, China · Department of Automation, BNRist, Tsinghua University, Beijing 100084, China

Abstract

Parameter-efficient fine-tuning (PEFT) has recently emerged as a pivotal research direction for adapting pre-trained point cloud transformers to diverse downstream tasks. Although existing methods achieve excellent fine-tuning performance with high parameter efficiency, they ignore inference efficiency. To tackle this problem, a novel PEFT method termed position anchor tuning (PAT) is proposed in this paper. As multi-head attention (MHA) and feed-forward network (FFN) are computation-heavy blocks in pre-trained transformers, PAT decreases their computational cost through token aggregation-expansion pairs. Each pair comprises a token aggregation module (TAM) and a token expansion module (TEM). For MHA and FFN blocks, TAMs extract representative tokens from their input tokens based on position anchors in 3D space. These extracted tokens, rather than the original input tokens, are processed by the blocks, thereby reducing the number of tokens involved in computation. Then, TEMs propagate the learned representations back to the original input tokens. Since TAMs are solely responsible for capturing task-specific representations, base-sharing low-rank adaptation (BSLoRA) is further introduced to enable them to learn such representations effectively with only a small number of trainable parameters. Extensive experiments on widely used benchmarks demonstrate that PAT performs comparably to state-of-the-art methods while incurring significantly lower computational overhead and fewer trainable parameters.

Figures & tables

Fig. 1: Inference differences between PAT and existing 3D PEFT methods in MHA/FFN blocks.

p

n

, and

m

(

m<n

) represent the number of prompt tokens, patch tokens, and extracted tokens, respectively. Existing 3D PEFT methods include two cases: increasing the number of input tokens (Case 1) and keeping the input tokens unchanged (Case 2).

Fig. 2: Illustration of PAT. For the convenience of presentation, the 3D spatial information required by TAMs and TEMs, including point patch positions $\{{{\bf{P}}_{i}}\}_{i=1}^{n}$ , position anchors $\{{{\bf{A}}_{i}}\}_{i=1}^{m}$ , and the attention maps $\{{\bf{A}}{{\bf{M}}_{i}}\}_{i=1}^{m}$ between position anchors and their neighbors, is processed within the PI module. TAMs, inserted before MHA or FFN blocks, extract $m$ anchor tokens from $n$ patch tokens based on the provided information, and then apply BSLoRA to effectively adapt these anchor tokens to downstream tasks in a parameter-efficient way.

Fig. 3: Parameter reduction (%) achieved by BSLoRA compared to LoRA for varying values of

b

Pre-trained model	Fine-tuning strategy	Reference	#Param. (M)	#GFLOPs	ScanObjectNN			ModelNet40
Pre-trained model	Fine-tuning strategy	Reference	#Param. (M)	#GFLOPs	OBJ_BG	OBJ_ONLY	PB_T50_RS	ModelNet40
Point-MAE [ 8 ]	Full		22.1 (100%)	4.76(-)	90.02	88.29	85.18	93.2
	IDPT [ 19 ]	ICCV 23	1.7 (7.69%)	7.1( $\uparrow$ 49.16% )	91.22 ( +1.20 )	90.02 ( +1.73 )	84.94( -0.24 )	93.3( +0.1 )
	DAPT [ 23 ]	CVPR 24	1.1 (4.97%)	5.0( $\uparrow$ 5.04% )	90.88( +0.86 )	90.19 ( +1.90 )	85.08( -0.10 )	93.5 ( +0.3 )
	Point-PEFT [ 47 ]	AAAI 24	0.7 (3.13%)	7.6( $\uparrow$ 37.37% )	89.67( -0.35 )	88.98( +0.69 )	84.91( -0.27 )	93.3( +0.1 )
	PointGST [ 18 ]	TPAMI 25	0.6 (2.77%)	4.8 ( $\uparrow$ 0.84% )	91.74 ( +1.72 )	90.19 ( +1.90 )	85.29( +0.11 )	93.5 ( +0.3 )

TABLE I: Classification accuracy (%) on the ModelNet40 dataset and three variants of the ScanObjectNN dataset. #Param. (M) denotes the average number of trainable parameters in millions. #GFLOPs denotes the average inference GFLOPs of backbones. Among 3D PEFT methods, the best results are in bold and the second-best results are underlined.

Method	Reference	5-way		10-way
Method	Reference	10-shot	20-shot	10-shot	20-shot
Pre-trained Point-MAE backbone
+ Full [ 8 ]	ECCV 22	96.3 $\pm$ 2.5	97.8 $\pm$ 1.8	92.6 $\pm$ 4.1	95.0 $\pm$ 3.0
+ IDPT [ 19 ]	ICCV 23	97.3 $\pm$ 2.1	97.9 $\pm$ 1.1	92.8 $\pm$ 4.1	95.4 $\pm$ 2.9
+ DAPT [ 23 ]	CVPR 24	96.8 $\pm$ 1.8	98.0 $\pm$ 1.0	93.0 $\pm$ 3.5	95.5 $\pm$ 3.2
+ Point-PEFT [ 47 ]	AAAI 24	95.5 $\pm$ 2.9	97.6 $\pm$ 1.7	91.7 $\pm$ 4.3	94.7 $\pm$ 3.0

TABLE II: Few-shot classification results on the ModelNet40 dataset. Results are reported as mean accuracy (%)

\pm

standard deviation (%) over 10 independent experiments.

Method	Reference	$\Delta G$	Params. (M)	CmIoU(%)	ImIoU(%)
Pre-trained Point-MAE backbone
+ Full [ 8 ]	ECCV 22	-	27.06	84.19	86.1
+ IDPT [ 19 ]	ICCV 23	$\uparrow$	5.69	83.79	85.7
+ DAPT [ 23 ]	CVPR 24	$\uparrow$	5.65	84.01	85.7
+ Point-PEFT [ 47 ]	AAAI 24	$\uparrow$	5.62	83.20	85.2
+ PointGST [ 18 ]	TPAMI 25	$\uparrow$	5.59	83.81	85.8

TABLE III: Part segmentation performance on the ShapeNetPart dataset. CmIoU (%) and ImIoU (%) denote the class-mean and instance-mean mIoU, respectively.

Notation	Local Information	Base-Sharing	SRT	Accuracy (%)
PAT_L&B&S	$\times$	$\times$	$\times$	74.91
PAT_L&S	$\times$	$\checkmark$	$\times$	80.81
PAT_L	$\times$	$\checkmark$	$\checkmark$	82.89
PAT_B&S	$\checkmark$	$\times$	$\times$	75.99
PAT_S	$\checkmark$	$\checkmark$	$\times$	84.14
PAT	$\checkmark$	$\checkmark$	$\checkmark$	85.46

TABLE IV: Ablation experiments on the PB_T50_RS dataset.

Fig. 4: Performance verification of BSLoRA.

Method	Shape Classification						Few-Shot Learning
Method	#Param. (M)	#GFLOPs	OBJ_BG	OBJ_ONLY	PB_T50_RS	ModelNet40	5w10s	5w20s	10w10s	10w20s
Full	360.5 (100%)	67.71(-)	97.2	96.6	93.4	94.1	98.0 $\pm$ 1.9	99.0 $\pm$ 1.0	91.1 $\pm$ 3.3	96.1 $\pm$ 2.8
IDPT [ 19 ]	10.0 (2.77%)	75.19( $\uparrow$ 11.05% )	98.11( +0.91 )	96.04( -0.56 )	92.99( -0.41 )	93.4( -0.7 )	96.8 $\pm$ 2.0	98.3 $\pm$ 1.4	95.3 $\pm$ 2.5	96.5 $\pm$ 2.5
DAPT [ 23 ]	4.2 (1.17%)	71.64( $\uparrow$ 5.80% )	98.11( +0.91 )	96.21( +0.39 )	93.02( -0.38 )	94.2( +0.1 )	97.2 $\pm$ 3.4	98.4 $\pm$ 1.3	95.9 $\pm$ 2.7	97.3 $\pm$ 2.4
Point-PEFT [ 47 ]	3.1 (0.86%)	73.62( $\uparrow$ 8.73% )	96.39( -0.81 )	94.66( -1.94 )	92.85( -0.55 )	93.5( -0.6 )	95.7 $\pm$ 2.5	98.0 $\pm$ 1.8	94.8 $\pm$ 2.7	96.3 $\pm$ 2.4
PointGST [ 18 ]	2.4 (0.67%)	67.98 ( $\uparrow$ 0.40% )	98.97 ( +1.77 )	97.59 ( +0.99 )	94.83 ( +1.43 )	94.8 ( +0.7 )	97.4 $\pm$ 2.0	99.6 $\pm$ 0.5	96.3 $\pm$ 3.6	97.6 $\pm$ 2.9

TABLE V: Classification accuracy (%) of different fine-tuning methods using the pre-trained PointGPT-L backbone. “

x

y

s” denotes the

x

-way

y

-shot setting.

Method	${\Delta G}$	Params. (M)	CmIoU(%)	ImIoU(%)
Full	-	339.39	84.80	86.6
IDPT [ 19 ]	$\uparrow$	33.17	82.91	85.1
DAPT [ 23 ]	$\uparrow$	33.85	83.03	85.2
Point-PEFT [ 47 ]	$\uparrow$	31.94	82.34	84.9
PointGST [ 18 ]	$\uparrow$	31.85	83.91	85.8
PAT	$\downarrow$ 12.0%	31.56	83.96	85.6

TABLE VI: Part segmentation results of different fine-tuning methods using the pre-trained PointGPT-L backbone.

Fig. 5: Comparisons of mAcc and mIoU on the S3DIS dataset.

Fig. 6: Comparisons of the number of trainable parameters and the inference GFLOPs of the backbone on the S3DIS dataset.

Fig. 7: Comparisons of AP25 and AP50 on the ScanNetV2 dataset.

Fig. 8: Comparisons of the number of trainable parameters and the inference GFLOPs of the backbone on the ScanNetV2 dataset.

Fig. 9: Inference speed comparisons with varying batch sizes.

Fig. 10: Comparisons of GPU memory at a batch size of 8.

Method	Design for	${\Delta G}$	Params. (M)	Accuracy(%)
Full		-	22.1	85.18
Adapter [ 16 ]	NLP	$\uparrow$	0.9	83.93
Prefix tuning [ 27 ]	NLP	$\uparrow$	0.7	77.72
LoRA [ 12 ]	NLP	-	0.9	81.74
DEPT [ 15 ]	NLP	$\uparrow$	0.3	79.70
FourierFT [ 61 ]	NLP	-	0.3	78.57

TABLE VII: Comparisons with representative PEFT methods designed for NLP and 2D vision on the PB_T50_RS dataset.

Parameter-Efficient Fine-Tuning

Parameter-efficient fine-tuning adapts pretrained models by updating small parameter subsets or lightweight modules instead of all weights.

See all 605 papers

Point Ladder Tuning: Parameter-Efficient Hierarchical Adaptation for 3D Point Cloud Understanding
Jul 21, 2026Junlin Chang, Longhao Zou, Rui Li3D Point Cloud Segmentation Point Cloud Classification
Beihang University, Beijing · Pengcheng Laboratory, Shenzhen
Mantis: Mamba-native Tuning is Efficient for 3D Point Cloud Foundation Models
May 5, 2026Zihao Guo, Yiding Sun, Jihua Zhu +33D Foundation Models Mamba
Xi’an Jiaotong University, Xi’an, China · Peking University, Beijing, China · School of Artificial Intelligence and Robotics, Hunan University, China +1
LitePT: Lighter Yet Stronger Point Transformer
Dec 15, 2025Yuanwen Yue, Damien Robert, Jianyuan Wang +4Point Cloud Learning Point Transformer
ETH Zurich · University of Oxford · University of Zurich

Jul 21, 2026J/K move · Enter open · S save

Point Ladder Tuning: Parameter-Efficient Hierarchical Adaptation for 3D Point Cloud Understanding

Junlin Chang, Longhao Zou, Rui Li

Beihang University, Beijing · Pengcheng Laboratory, Shenzhen

3D Point Cloud Segmentation Point Cloud Classification Multi-Scale Feature Fusion Prompt Tuning Point Cloud Understanding Parameter-Efficient Fine-Tuning

Open paper AskPDF arXiv

Fine-tuning pre-trained point-cloud backbones typically updates all parameters, resulting in substantial computation and memory overhead. More importantly, modern point backbones rely on aggressive tokenization and downsampling, which yields compact global tokens but irreversibly discards fine-grained local geometry, an inherent bottleneck for parameter-efficient adaptation. Consequently, existing PEFT methods that operate only on these coarsened tokens can modulate global semantics but struggle to recover the missing multi-scale locality. We present Point Ladder Tuning (PLT), a locality-aware PEFT framework that performs hierarchical, instance-conditioned adaptation while keeping the backbone frozen. PLT forms a lightweight closed loop: (i) a Hierarchical Ladder Network (HLN) constructs a multi-resolution local feature pyramid directly from raw points; (ii) a Local-Global Fusion (LGF) aligns and fuses local pyramids with intermediate backbone semantics; and (iii) a Dynamic Prompt Generator produces instance-aware multi-scale prompts to modulate the frozen backbone effectively. For dense prediction, we further introduce a lightweight segmentation head that progressively upsamples fused features and leverages backbone priors to refine fine structures. Extensive experiments on classification and dense prediction show that PLT consistently surpasses prior PEFT baselines with minimal tunable parameters. PLT achieves state-of-the-art performance using only 2.71% trainable parameters for classification and 7.69% for dense prediction, and scales favorably to larger backbones, requiring merely 0.36% parameters on PointGPT-L. The code is released at https://github.com/JunLinChang/ECCV2026-PLT.

Abstract

Parameter-Efficient Fine-Tuning

Explore similar work

Point Ladder Tuning: Parameter-Efficient Hierarchical Adaptation for 3D Point Cloud Understanding

Mantis: Mamba-native Tuning is Efficient for 3D Point Cloud Foundation Models

LitePT: Lighter Yet Stronger Point Transformer