cs.CVApr 3, 2026

Deformation-based In-Context Learning for Point Cloud Understanding

Authors: Chengxing Lin, Jinhong Deng, Yinjie Lei, Wen Li

Organizations: Shenzhen Institute for Advanced Study, UESTC · School of Computer Science and Engineering, UESTC · Sichuan University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Recent advances in point cloud In-Context Learning (ICL) have demonstrated strong multitask capabilities. Existing approaches typically adopt a Masked Point Modeling (MPM)-based paradigm for point cloud ICL. However, MPM-based methods directly predict the target point cloud from masked tokens without leveraging geometric priors, requiring the model to infer spatial structure and geometric details solely from token-level correlations via transformers. Additionally, these methods suffer from a training-inference objective mismatch, as the model learns to predict the target point cloud using target-side information that is unavailable at inference time. To address these challenges, we propose DeformPIC, a deformation-based framework for point cloud ICL. Unlike existing approaches that rely on masked reconstruction, DeformPIC learns to deform the query point cloud under task-specific guidance from prompts, enabling explicit geometric reasoning and consistent objectives. Extensive experiments demonstrate that DeformPIC consistently outperforms previous state-of-the-art methods, achieving reductions of 1.6, 1.8, and 4.7 points in average Chamfer Distance on reconstruction, denoising, and registration tasks, respectively. Furthermore, we introduce a new out-of-domain benchmark to evaluate generalization across unseen data distributions, where DeformPIC achieves state-of-the-art performance.

Figures & tables

Figure 2: Overall framework of DeformPIC. Our model consists of two core components. The Deformation Extraction Network (DEN) extracts the geometric transformation from an example pair (prompt input→prompt target) into a task embedding. This embedding then modulates the Deformation Transfer Network (DTN), thereby guiding it to apply the same transformation to a new query input to produce the final prediction. The entire network is trained by minimizing the Chamfer Distance [ 9 ] against the ground truth.
ModelsVenuesPart.Reconstruction CD ↓\downarrowDenoising CD ↓\downarrowRegistration CD ↓\downarrow
mIoU ↑\uparrowL1L2L3L4L5Avg.L1L2L3L4L5Avg.L1L2L3L4L5Avg.
Task-Specific Models: trained separately for single task
PointNet [ 32 ]CVPR’1777.53.73.73.83.94.13.94.14.04.14.04.24.15.35.96.97.78.56.9
DGCNN [ 44 ]TOG’1976.13.93.94.04.14.34.04.74.54.64.54.74.66.26.77.37.47.77.1
PCT [ 14 ]CVM’2179.52.42.42.52.63.02.62.32.22.22.22.32.25.35.76.36.97.26.3
ACT [ 8 ]ICLR’2381.22.42.52.32.52.82.52.22.32.22.32.52.35.15.65.96.07.05.9
Table 1: Comparison with state-of-the-art methods on the ShapeNet In-Context dataset [ 10 ] . For reconstruction, denoising, and registration, we report Chamfer Distance [ 9 ] loss ( ×1000\times 1000 ). For part segmentation (Part.), we report mIoU. Since PIC [ 10 ] does not provide the details of specific multi-task heads, we tested with our own designed multi-task heads, which are marked with †\dagger . Bold: the best results. Underline: the second-best results.
Figure 3: Qualitative comparison on ShapeNet In-Context dataset. From left to right: input point clouds, PIC-Cat [ 10 ] results, our DeformPIC results, and ground truth. Our method achieves better geometric detail preservation, improved structural coherence, and fewer visual artifacts across diverse tasks.
ModelsRec. CD ↓\downarrowDen. CD ↓\downarrowReg. CD ↓\downarrow
M.S.M.S.M.S.
Multitask Models
Point-MAE [ 30 ]5.05.38.19.638.226.1
ReCon [ 33 ]5.75.89.510.937.941.9
PCP-MAE [ 54 ]5.76.09.811.237.642.8
UniPre3D [ 46 ]5.55.612.614.838.742.9
Table 2: Comparison of generalization ability with state-of-the-art methods on the ModelNet40 In-Context (M.) and ScanObjectNN In-Context (S.) datasets. We report task-averaged Chamfer Distance [ 9 ] ( ×1000\times 1000 ).
ModelConsis.DTNDENRec.Den.Reg.
PIC-Cat✗✗✗4.35.314.1
✓✗✗3.24.03.5
Ours✓✓✗3.13.82.9
✓✓✓2.73.52.0
Table 3: Ablation Study of DeformPIC on ShapeNet In-Context. We report the task-averaged Chamfer Distance ( ×1000\times 1000 ) for the reconstruction, denoising, and registration tasks.
Figure 4: Task Feature Visualization of DeformPIC on the ShapeNet / ModelNet40 / ScanObjectNN In-Context datasets. We use t-SNE [ 29 ] to reduce the dimensionality of the task features to a 2D space and visualize the distributions of task features.
Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

ModelsFlops(G)Param.(M)Reconstruction CD ↓\downarrowDenoising CD ↓\downarrowRegistration CD ↓\downarrow
L1L2L3L4L5Avg.L1L2L3L4L5Avg.L1L2L3L4L5Avg.
PIC-Cat [ 10 ]11.3128.913.23.64.64.95.54.33.94.65.36.06.85.310.011.413.816.918.614.1
PIC-Sep [ 10 ]8.1428.914.74.34.34.45.74.76.37.27.98.28.67.68.69.210.211.312.410.3
DeformPIC- 12d5.3340.482.12.22.42.73.92.72.83.33.53.73.83.41.81.81.91.92.01.9
DeformPIC- 8d4.8729.842.22.32.52.83.92.72.83.33.63.83.93.51.91.91.92.02.12.0
Appendix
Table 4: Comparison of model variants in terms of computational complexity, and in-context performance in ShapeNet In-Context [ 10 ] .
ModelsRec.Den.Reg.
F-score@ τ\tauF-score@ τ\tauF-score@ τ\tau
PIC-Cat [ 10 ]38.3633.0626.98
PIC-Sep [ 10 ]42.7832.8641.57
DeformPIC- 8d57.9849.9566.04
Appendix
Table 5: Comparison with state-of-the-art methods in F-score@ τ\tau ( τ=0.001\tau=0.001 ) on ShapeNet In-Context [ 10 ] .
ModelsRec. EMDDen. EMDReg. EMD
PIC-Cat [ 10 ]56.3052.8975.14
PIC-Sep [ 10 ]60.8365.2764.28
DeformPIC- 8d37.0147.3224.05
Appendix
Table 6: Comparison with state-of-the-art methods in EMD ( ×1000\times 1000 ) on ShapeNet In-Context [ 10 ] .
Encoding StrategiesRec. CD ↓\downarrowDen. CD ↓\downarrowReg.CD ↓\downarrow
Static ( Predict Task ID )3.24.09.9
Dynamic ( Predict Task Token )2.73.52.0
Appendix
Table 7: Static vs. Dynamic Encoding on DEN.
Figure 5: Visualization results on registration task compared with the PIC [ 10 ] and our approach.
Figure 6: Visualization results on reconstruction task compared with the PIC [ 10 ] and our approach.
Figure 7: Visualization results on denoising task compared with the PIC [ 10 ] and our approach.
Figure 8: Visualization results on part segmentation task compared with the PIC [ 10 ] and our approach.
More in this topic

Point Cloud Denoising

See all 10 papers

Explore similar work

CardsList
May 26, 2026

I2PRef: Image-Driven Point Completion with Iterative Refinement

We present an image-conditioned point cloud completion approach that treats images as the primary geometric source rather than a secondary guide. To this end, we introduce an Image-to-Point (I2P) module that can reconstruct complete point clouds directly from a single RGB image, with no need for 3D inputs. Additionally, we introduce a transformer-based Point-to-Point (P2P) refinement module that uses self- and cross-attention between point tokens and image features to iteratively refine the coarse I2P output. The I2P module enables the image encoder to learn rich geometric representations, while the P2P module progressively recovers fine-grained details. Unlike existing multimodal methods that rely on auxiliary losses or fusion modules, our explicit I2P task provides a strong, geometry-aware prior based on images alone. Extensive experiments on ShapeNet-ViPC demonstrate state-of-the-art completion performance with a 12.3% relative Chamfer Distance improvement over prior methods. Code is available at: https://github.com/AzharSindhi/I2PRef.git

Azhar Hussian, Marina Ritthaler, André Kaup +1

Friedrich-Alexander-Universität Erlangen-Nürnberg

Point Cloud Completion3D Reconstruction3D Shape CompletionSingle-View 3D Reconstruction
AskPDF
Mar 11, 2026

Pointy - A Lightweight Transformer for Point Cloud Foundation Models

Foundation models for point cloud data have recently grown in capability, often leveraging extensive representation learning from language or vision. In this work, we take a more controlled approach by introducing a lightweight transformer-based point cloud architecture. In contrast to the heavy reliance on cross-modal supervision, our model is trained only on 39k point clouds - yet it outperforms several larger foundation models trained on over 200k training samples. Interestingly, our method approaches state-of-the-art results from models that have seen over a million point clouds, images, and text samples, demonstrating the value of a carefully curated training setup and architecture. To ensure rigorous evaluation, we conduct a comprehensive replication study that standardizes the training regime and benchmarks across multiple point cloud architectures. This unified experimental framework isolates the impact of architectural choices, allowing for transparent comparisons and highlighting the benefits of our design and other tokenizer-free architectures. Our results show that simple backbones can deliver competitive results to more complex or data-rich strategies. The implementation, including code, pre-trained models, and training protocols, is available at https://github.com/KonradSzafer/Pointy.

Konrad Szafer, Marek Kraft, Dominik Belter

Institute of Robotics and Machine Intelligence, Poznan University of Technology

Point Cloud Learning3D Foundation Models3D Representation Learning3D ViTs
AskPDF
May 24, 2026

PQDT: Pseudo-Query Dual Transformer for Robust Point Cloud Restoration

Point clouds are a fundamental 3D representation in computer vision, enabling a wide range of perception tasks. However, real-world point clouds often suffer from degradations such as incompleteness, noise, outliers, and irregular density, caused by sensor limitations or occlusions. Recovering clean and detailed shapes from such degraded data is crucial for downstream applications. While existing learning-based methods achieve progress on individual tasks like completion or denoising, they typically rely on global bottleneck features, which lose fine-grained geometry and remain sensitive to varying input quality. We propose a unified 3D restoration network that directly takes point clouds as input and adaptively reconstructs high-quality geometry under diverse degradation scenarios. At the core of our approach is a Pseudo-Query module, implemented within a Transformer backbone, which reformulates geometric translation into two cooperative stages to enhance structural clarity, robustness, and local detail preservation. Extensive experiments on curated benchmarks demonstrate that our approach surpasses state-of-the-art performance in general 3D restoration. It effectively handles complex combinations of completion, deformation, and denoising degradations. With this work, we provide a novel unified, point-only backbone for robust 3D restoration, enabling more versatile 3D perception.

Haoqing Wu, Alexa Nawotki, Jochen Garcke

Mercedes-Benz AG, Germany · University of Bonn, Germany · Fraunhofer SCAI, Germany

Point Cloud Completion3D ViTs3D ReconstructionPoint Cloud DenoisingPoint Cloud Learning
AskPDF