LeafInst - Unified Instance Segmentation Network for Fine-Grained Forestry Leaf Phenotype Analysis: A New UAV based Benchmark

Taige Luo; Junru Xie; Chenyang Fan; Bingrong Liu; Ruisheng Wang; Yang Shao; Sheng Xu; Lin Cao

cs.CVMar 4, 2026

LeafInst - Unified Instance Segmentation Network for Fine-Grained Forestry Leaf Phenotype Analysis: A New UAV based Benchmark

Authors: Taige Luo, Junru Xie, Chenyang Fan, Bingrong Liu, Ruisheng Wang, Yang Shao, Sheng Xu, Lin Cao

Organizations: College of Information Science and Technology & Artificial Intelligence, Nanjing Forestry University, Nanjing, 210037, Jiangsu, China · Department of Geography, College of Natural Resources and Environment, Virginia Tech, Blacksburg, VA, 24061, USA · School of Software, Hefei University of Technology, Hefei, 230601, Anhui, China · School of Architecture & Urban Planning, Shenzhen University, Shenzhen, 518060, Guangdong, China · Co-Innovation Center for Sustainable Forestry in Southern China, Nanjing Forestry University, Nanjing, 210037, Jiangsu, China

Abstract

Intelligent forest tree breeding has advanced plant phenotyping, yet existing research largely focuses on large-leaf agricultural crops, with limited attention to fine-grained leaf analysis of sapling trees in open-field environments. Natural scenes introduce challenges including scale variation, illumination changes, and irregular leaf morphology. To address these issues, we collected UAV RGB imagery of field-grown saplings and constructed the Poplar-leaf dataset, containing 1,202 branches and 19,876 pixel-level annotated leaf instances. To our knowledge, this is the first instance segmentation dataset specifically designed for forestry leaves in open-field conditions. We propose LeafInst, a novel segmentation framework tailored for irregular and multi-scale leaf structures. The model integrates an Asymptotic Feature Pyramid Network (AFPN) for multi-scale perception, a Dynamic Asymmetric Spatial Perception (DASP) module for irregular shape modeling, and a dual-residual Dynamic Anomalous Regression Head (DARH) with Top-down Concatenation decoder Feature Fusion (TCFU) to improve detection and segmentation performance. On Poplar-leaf, LeafInst achieves 68.4 mAP, outperforming YOLOv11 by 7.1 percent and MaskDINO by 6.5 percent. On the public PhenoBench benchmark, it reaches 52.7 box mAP, exceeding MaskDINO by 3.4 percent. Additional experiments demonstrate strong generalization and practical utility for large-scale leaf phenotyping.

Figures & tables

Figure 1: Challenges in forestry leaf phenotyping. (a) Scale changes, (b) Brightness changes, (c) Shape changes.

Figure 2: The workflow and downstream applications of the proposed framework LeafInst.

Figure 3: Overview of the main structure of the proposed LeafInst.

Figure 4: Adaptive spatial fusion operation (ASFF) built in AFPN.

Figure 5: The workflow of dynamic anomalous regression head.

Figure 6: The feature visualization maps of dynamic asymmetric spatial perception.

Figure 7: Different feature fusion strategies. (a) Top-down Concatenation - decoder Feature Fusion (b) Top-down Pixel-wise Addition Feature Fusion.

Figure 8: Study area and dataset production.

Figure 9: Examples of different scenarios. (a) Forestry Dataset: Poplar-leaf, (b) Agriculture Dataset: PhenoBench.

Figure 10: Qualitative comparison results with different models on Poplar-leaf

Figure 11: Qualitative comparison results with different models on Phenobench.

Methods	seg/mAP	seg/AP50	seg/AP75	box/mAP	box/AP50	box/AP75
Queryinst	56.9	76.4	62.1	55.3	75.3	60.2
CascadeRcnn	63.6	84.7	72.3	64.1	84.6	71.4
MaskRcnn	61.8	86.5	68.5	61.5	86.4	67.5
Solo	56.8	80.9	63.0	–	–	–
SoloV2	65.3	86.7	71.6	–	–	–
PointRend	64.4	86.5	70.5	62.2	86.4	69.4

Table 1: Poplar-Leaf Validation Results (%): x x : 1st place, x x : 2nd place, x x : 3rd place.

Methods	seg/mAP	seg/AP50	seg/AP75	box/mAP	box/AP50	box/AP75
Queryinst	58.2	79.7	64.4	55.7	79.2	59.5
CascadeRcnn	62.3	83.0	69.3	62.4	83.0	69.8
MaskRcnn	62.1	90.7	68.8	61.2	89.8	65.2
Solo	57.9	84.3	65.1	–	–	–
SoloV2	64.9	88.6	71.7	–	–	–
PointRend	63.6	85.0	70.5	61.7	84.6	68.2

Table 2: Poplar-Leaf Test Results (%): x x : 1st place, x x : 2nd place, x x : 3rd place.

Methods	box/mAP	box/AP50	seg/mAP	seg/AP50
LeafInst	52.7	82.3	50.2	80.5
YOLOv8	48.6	78.4	45.7	76.0
YOLOv11	47.5	77.7	45.6	75.9
MaskDINO	49.3	80.2	51.0	80.3

Table 3: Quantitative Comparison of Different Methods on PhenoBench (%): x x : 1st place, x x : 2nd place.

Figure 12: Visualization comparison with Visual Foundation Models. (a) Input (b) LeafInst (c) SAM (d) Fast-SAM (e) semantic-SAM

Figure 13: Accuracy and convergence analysis.

Figure 14: Segmentation results of different conditions by LeafInst.

Figure 15: Schematic diagram of phenotypic indicators - taking Leaf#306 as a example.

Method	F1-Acc	F2-Acc	F3-Acc	F4-Acc	F5-Acc
LeafInst	90.9	86.6	63.1	88.8	66.8
SAM	81.8	28.8	18.4	22.2	–
FastSAM	63.6	78.5	32.0	72.2	61.1
semantic-SAM	86.3	67.3	56.3	87.5	71.0

Table 4: Manual visual inspection accuracy (%) under different evaluation settings.

Size	Baseline	AFPN	DSAP(w/o)	DARH	TAFU	TCFU	Weight	seg/mAP	box/mAP
512	✓				✓			64.5	59.6
1024	✓				✓			65.3	60.2
1024	✓	✓			✓			65.6	61.7
1024	✓	✓	✓		✓			64.5	60.6
1024	✓	✓	✓	✓	✓		✓	67.7	64.6
1024	✓	✓	✓	✓		✓	✓	68.4	65.8

Table 5: Ablation study of different modules.

Prediction (Pred)
ID	width	height	perimeter	area	round	rect	$R_{m}$	$G_{m}$	$B_{m}$	$\bar{R}$	$\bar{G}$	$\bar{B}$
1	294.00	386.00	1135.99	78915.50	0.77	0.74	43.00	71.00	12.00	44.18	71.65	13.99
2	296.00	299.00	1008.25	60085.50	0.74	0.69	44.00	75.00	10.00	46.94	76.24	12.38
3	176.00	370.00	964.91	48527.50	0.65	0.75	32.00	62.00	6.00	34.16	62.99	8.81
4	173.00	208.00	640.84	26074.00	0.80	0.75	79.00	104.00	12.00	80.73	104.55	12.93
5	132.00	218.00	611.06	17917.50	0.60	0.77	37.00	64.00	11.00	40.81	65.70	13.78

Table 6: Prediction (pred) and Ground Truth (gt) Indicators - taking Leaf#306 as a example

Figure 16: Regression curves of predicted and GT secondary Indicators.

$R_{m}$	$\bar{R}$	$R_{1/3}$	$R_{2/3}$	$G_{m}$	$\bar{G}$	$G_{1/3}$	$G_{2/3}$	$B_{m}$	$\bar{B}$	$B_{1/3}$	$B_{2/3}$
80.02	82.50	73.71	87.15	111.68	113.10	105.49	118.38	41.39	44.45	34.40	49.36

Table 7: The optimal value of intermediate indicators of colors

Shape Indicators
	width	height	perimeter	area	roundness	rectangularity
$R^{2}$	0.891	0.864	0.791	0.901	0.475	0.534
RMSE	29.225	36.992	149.790	7554.338	0.111	0.059
MAE	11.370	14.861	72.894	2269.508	0.072	0.037

Table 8: Statistical table of intermediate indicators.

R_{m}

denotes the median value,

\bar{R}

denotes the mean value,

R_{1/3}

denotes the lower third quartile,

R_{2/3}

denotes the higher third quartile. Other channels are same as R channel.

Shape Indicators Weights
width	height	perimeter	area	roundness	rectangularity
0.15	0.11	0.13	0.30	0.06	0.03

Table 9: Indicator weights of LGCI (keep two decimals)

Figure 17: Quantitative results of the Leaf Growth Condition Indicator (LGCI) scores. (a, d) Input images; (b, e) instance segmentation results produced by LeafInst (IoU > 0.3); (c, f) LGCI score visualization maps. Red and blue boxes highlight leaf instances exhibiting relatively favorable (positive) and unfavorable (negative) morphological characteristics, respectively

Explore similar work

ReLeaf: Benchmarking Leaf Segmentation across Domains and Species
May 5, 2026Robert Martinko, Daniel Steininger, Julia Simon +2Agricultural Image Analysis Instance Segmentation
1AIT Austrian Institute of Technology, Center for Vision, Automation & Control · University of Applied Sciences Technikum Wien, Computer Science & Applied Mathematics
Highly Detailed and Generalizable Broadleaf Tree Crown Instance Segmentation from UAV Imagery
May 15, 2026Mitsutaka Nakada, Takahiko Ikebata, Kengo Ikebata +7Image Segmentation Remote Sensing Image Segmentation
DeepForest Technologies Co., Ltd · DeepForest Technologies Co., Ltd., Kyoto 600-8006, Japan · YM Lab +9
SilvaScenes: Tree Detection and Species Classification from Under-Canopy Images in Natural Forests
Oct 10, 2025David-Alexandre Duclos, William Guimont-Martin, Gabriel Jeanson +7Instance Segmentation Tree Species Classification
Northern Robotics Laboratory, Universit´e Laval, Qu´ebec, QC, G1V 0A6, Canada · D´epartement des sciences du bois et de la forˆet, Universit´e Laval, Qu´ebec, QC, G1V 0A6, Canada · Institut des Sciences de la Forˆet temp´er´ee, Universit´e du Qu´ebec en Outaouais, Ripon, QC, J0V 1V0, Canada

May 5, 2026J/K move · Enter open · S save

ReLeaf: Benchmarking Leaf Segmentation across Domains and Species

Robert Martinko, Daniel Steininger, Julia Simon, Andreas Trondl+1

1AIT Austrian Institute of Technology, Center for Vision, Automation & Control · University of Applied Sciences Technikum Wien, Computer Science & Applied Mathematics

Agricultural Image Analysis Instance Segmentation Cross-Domain Generalization Precision Agriculture

Open paper AskPDF arXiv

Rising global food demand and growing climate pressure increase the need for sustainable, precise agricultural practices. Automated, individualized plant treatment relies on fine-grained visual analysis, yet leaf-level segmentation remains underexplored despite its value for assessing crop health, growth dynamics, yield potential and localized stress symptoms. Progress is limited by a lack of dedicated datasets, especially regarding species coverage, and by the absence of systematic evaluations of modern instance-segmentation architectures for this task. We address these gaps by surveying current data and identifying four suitable, publicly available leaf-segmentation datasets. Using them, we compare one-stage, two-stage and Transformer-based detectors and identify a YOLO26 model configuration to provide the best trade-off for real-world precision-agriculture tasks. Extensive cross-domain generalization experiments reveal substantial performance drops across plant species and recording setups, especially for models trained solely on laboratory data. To strengthen data availability, we introduce a new benchmark dataset with leaf-level masks for 23 plant species, created via semi-automatic annotation of selected CropAndWeed images. A model trained on all four existing datasets achieves a mean mAP50-95 of 83.9% across their corresponding test sets and 40.2% on our new benchmark, demonstrating improved generalization and highlighting the need for diverse leaf-segmentation datasets in robust precision agriculture.

Abstract

Explore similar work

ReLeaf: Benchmarking Leaf Segmentation across Domains and Species

Highly Detailed and Generalizable Broadleaf Tree Crown Instance Segmentation from UAV Imagery

SilvaScenes: Tree Detection and Species Classification from Under-Canopy Images in Natural Forests