PointRAFT: 3D deep learning for high-throughput prediction of potato tuber weight from partial point clouds
Authors: Pieter M. Blok, Haozhou Wang, Hyun Kwon Suh, Peicheng Wang, James Burridge, Wei Guo
Organizations: Graduate School of Agricultural and Life Sciences, The University of Tokyo, 1-1-1 Midori-cho, Nishitokyo-city, 188-0002, Tokyo, Japan · Department of Integrative Biological Sciences and Industry, Sejong University, 209 Neungdong-ro, 05006, Seoul, Republic of Korea
Key phenotypic traits, such as crop size, volume, and weight, can be estimated non-destructively from point clouds. In practice, however, point clouds of crops are typically incomplete due to self-occlusion, leading to systematic errors when estimating these traits. To overcome this limitation, crop traits can be inferred directly from incomplete point clouds using deep learning-based regression networks. These networks, however, may perform suboptimally, because their predictions do not incorporate prior geometric knowledge of the crop. Such geometric knowledge is often available in structured settings, such as on a potato harvester, where the fixed distance between the camera and the conveyor belt allows tuber height to be derived from the point cloud. To exploit this geometric cue, we introduce PointRAFT, a high-throughput point cloud regression network that incorporates object height through a dedicated height embedding. We hypothesized that this height embedding improves tuber weight estimation from partial point clouds. To test this hypothesis, we collected 26,688 point clouds of 859 potato tubers on an operational harvester, spanning four cultivars and three growing seasons. On a test set of 5,254 point clouds representing 172 tubers, PointRAFT achieved a mean absolute error (MAE) of 11.7 g and a root mean squared error (RMSE) of 17.1 g, outperforming six point cloud regression networks without a height embedding. Requiring only 6.3 ms per point cloud for preprocessing and inference, PointRAFT is fast enough for high-throughput weight estimation on a commercial potato harvester. Beyond potato weight estimation, PointRAFT provides a versatile regression network applicable to a wide range of 3D phenotyping and robotic perception tasks. The code, network weights, and a subset of the dataset are publicly available at https://github.com/pieterblok/pointraft.
Figures & tables
Figure 1: Schematic representation of the PointRAFT network, which directly regresses continuous 3D shape properties, such as tuber weight, from partial point clouds. Local geometric structures are captured by the set abstraction (SA) modules of PointNet++ (inset), which sample representative points, group their neighbors, and encode each neighborhood with a shared MLP. The architectural novelty of PointRAFT lies in the object height embedding, which provides an additional geometric cue for weight estimation.
Figure 2: (a) and (b) provide overviews of the first imaging system, which was installed on a single-row potato harvester and used during the 2023 growing season. (c) Inside the imaging box, an RGB-D camera was installed, together with four LED strips that provided the necessary illumination inside the box. The sides of the box were covered with a reflective curtain to diffuse the light evenly across the conveyor belt.
Figure 3: (a) Overview of the second imaging system, which was used during the 2024 and 2025 growing seasons. One side plate has been removed to reveal the internal components. (b) The yield monitoring system with enclosed sides during operation on the harvester. (c) On the ceiling plate of the enclosure, 24 LED strips and the RGB-D camera were installed.
Camera-belt
Potato
Sampled
Year
Field
Latitude
Longitude
distance [m]
cultivar
potatoes
Note
2023
1
42.610316
143.156753
0.35
Sayaka
280
2023
-
-
-
0.35
Kitahime
31
Indoor
2023
-
-
-
0.35
Corolle
30
Indoor
2023
-
-
-
0.35
Haruka
30
Indoor
2024
2
42.616872
143.174312
0.46
Kitahime
258
Table 1: Overview of the three potato fields and the indoor experiment from which the data was acquired.
Figure 4: Kernel density estimate plot for visualizing the weight distribution in the training, validation, and test set.
Potato
Point
Mean absolute error (root mean square error) [g] ↓
Subset comparison
tubers
clouds
PointNet
PointNet++
DGCNN
PointMLP
RandLA-Net
PTv3-C
PointRAFT
Potato cultivar
Sayaka
59
1,670
13.9 (19.3)
14.0 (18.9)
11.2 (15.1)
11.5 (16.1)
13.2 (18.4)
13.1 (17.8)
10.3 (14.8)
Kitahime
98
3,144
16.7 (23.1)
15.5 (21.6)
14.1 (19.4)
16.0 (23.3)
16.2 (21.9)
14.5 (20.8)
11.6 (16.7)
Corolle
8
222
23.7 (27.4)
24.0 (30.1)
14.8 (19.6)
14.6 (18.7)
18.8 (21.9)
13.4 (17.6)
15.5 (21.9)
Haruka
7
218
26.3 (31.0)
24.1 (28.9)
19.3 (23.7)
20.5 (25.9)
25.2 (29.8)
21.1 (24.4)
19.2 (29.7)
Growing season
2023
80
2,271
16.3 (21.8)
16.2 (21.5)
12.5 (16.8)
13.1 (18.1)
15.2 (20.4)
14.0 (18.6)
11.4 (17.3)
Table 2: Weight regression results of the seven point cloud regression networks per potato cultivar, growing season, and weight class, and for the entire test set. Results are given as the mean absolute error, with the root mean square error in parentheses. The lowest mean absolute error and the shortest processing time are printed in bold. PTv3-C denotes our compact adaptation of Point Transformer V3.
Figure 5: Predicted versus ground truth weights of the seven point cloud regression networks on the 5,254 test point clouds. Each panel reports the coefficient of determination (R²) and the bias, defined as the mean signed error. The dotted line indicates perfect agreement (x = y).
Figure 6: Mean absolute error (MAE) of the seven point cloud regression networks per horizontal image region, calculated over the entire test set (5,254 point clouds). The number of point clouds per region is given by n . The cell colors range from green (lowest MAE, 11.1 g) to red (highest MAE, 17.2 g) across all regions and networks, and the lowest MAE per region is printed in bold. The background image indicates the region boundaries; the tracked tuber in this example is the one with a red thumbtack in the 400–450 pixel region.
MAE [g] ↓
RMSE [g] ↓
Time [ms] ↓
Ablation
Category
abs
rel
abs
rel
abs
rel
PointRAFT
Baseline
11.7
-
17.1
-
6.3
-
1024 → 512
Input points
12.3
+5.8%
17.7
+3.1%
5.4
− 14.8%
1024 → 2048
Input points
12.1
+4.0%
18.0
+5.2%
9.1
+44.6%
No data augmentation
Network training
13.6
+16.8%
19.3
+12.6%
6.2
− 0.9%
Smooth L1 → L1
Loss function
12.3
+5.2%
17.8
+3.7%
6.2
− 1.7%
Table 3: Performance metrics and total inference time for the eleven studied ablations relative to the baseline performance of PointRAFT. The largest increase in MAE, RMSE and total inference time is printed in bold.
Laptop
PC
Edge device
Device
Lenovo Legion Pro
Alienware m15 R7
GALLERIA ZA9R
ZOTAC ZBOX
reComputer J4012
OS
Ubuntu 24.04
Windows 11
Manjaro Linux 6.18
Debian 13
JetPack 5.1.3
CPU
Intel i9-13900HX
AMD Ryzen 7 6800H
AMD Ryzen 9 5950X
Intel i7-7700HQ
8-core ARMv8
RAM
32 GB
16 GB
128 GB
16 GB
16 GB
GPU
RTX 4090 (M)
RTX 3070 Ti (M)
RTX 3090
GTX 1080
Jetson Orin
VRAM
16 GB
8 GB
24 GB
8 GB
Shared 16 GB
Table 4: Hardware configurations and mean stage-wise processing time and throughput.
Modern neural architectures for 3D point cloud processing contain both convolutional layers and attention blocks, but the best way to assemble them remains unclear. We analyse the role of different computational blocks in 3D point cloud networks and find an intuitive behaviour: convolution is adequate to extract low-level geometry at high-resolution in early layers, where attention is expensive without bringing any benefits; attention captures high-level semantics and context in low-resolution, deep layers more efficiently, where convolution inflates the parameter count. Guided by this design principle, we propose a new, improved 3D point cloud backbone that employs convolutions in early stages and switches to attention for deeper layers. To avoid the loss of spatial layout information when discarding redundant convolution layers, we introduce a novel, parameter-free 3D positional encoding, PointROPE. The resulting LitePT model has 3.6× fewer parameters, runs 2× faster, and uses 2× less memory than the state-of-the-art Point Transformer V3, but nonetheless matches or outperforms it on a range of tasks and datasets. Code and models are available at: https://github.com/prs-eth/LitePT.
Yuanwen Yue, Damien Robert, Jianyuan Wang +4
ETH Zurich · University of Oxford · University of Zurich
3D point cloud perception remains tightly coupled to custom CUDA operators for spatial operations, limiting portability and efficiency on non-NVIDIA, AMD, and embedded hardware. We introduce PointTransformerX (PTX), a fully PyTorch-native vision transformer backbone for 3D point clouds, removing all custom CUDA operators and external libraries while retaining competitive accuracy. PTX introduces 3D-GS-RoPE, a rotary positional embedding that encodes 3D spatial relationships directly in self-attention without neighborhood construction, and further replaces sparse convolutional patch embedding with a linear projection. PTX explores inference-time scaling of attention windows to improve accuracy without retraining. With a redesigned feed-forward network, PTX achieves 98.7% of PointTransformer V3's accuracy on ScanNet with 79.2% fewer parameters and executing 1.6\times faster while requiring just 253 MB memory. PTX runs natively on NVIDIA GPUs, AMD GPUs (ROCm), and CPUs, providing an efficient and portable foundation for point cloud perception.
Laurenz Reichardt, Nikolas Ebert, Oliver Wasenmüller
High-throughput phenotyping depends on accurate 3D reconstruction of plants across growth stages, yet the development and evaluation of temporal completion methods are limited by the lack of datasets with complete geometric ground truth. To address this challenge, we introduce SynthCrop4D, a procedurally generated synthetic dataset of temporally evolving plant point clouds that provides controllable noise, occlusion, and complete plant geometry for benchmarking reconstruction methods. Using this dataset, we evaluate a two-stage pipeline that combines spatial denoising and temporal point cloud completion. First, a denoising module removes structural artifacts from raw laser-scanned point clouds. The resulting data are then processed by an Adaptive Temporal PoinTr model that reconstructs the current growth stage (t) using information from the previous stage (t-1), enabling recovery of regions missing due to self-occlusion. We evaluate the proposed framework on both SynthCrop4D and the real-world Pheno4D dataset (tomato and maize) under settings with and without denoising. Results show that denoising substantially improves reconstruction quality, with the best configuration achieving a Chamfer Distance of 0.0061 on SynthCrop4D (Temporal PoinTr + Mamba-DG) and an F-Score of 0.2080 on Pheno4D (Vanilla PoinTr + Mamba-DG). We further demonstrate the use of completed point clouds for phenotypic trait extraction, including plant height, canopy width, and convex hull volume, obtaining hull-volume MAEs of 0.021 on synthetic data and 0.343 on real data. Together, SynthCrop4D and the proposed pipeline provide a benchmark and methodology for temporal plant reconstruction and high-throughput crop phenotyping.
Mrudul Mittal, Soumyashree Kar
Department of Electronics Engineering, Sardar Vallabhbhai National Institute of Technology, Surat, Gujarat, India · Centre of Studies in Resources Engineering, Indian Institute of Technology Bombay, Mumbai, Maharashtra, India