Latent Commonality Expectation-Maximisation for Box-supervised Tree Crown Instance Segmentation

Thomas Pitts; Kunqi Li; Bin Liang

cs.CVSep 22, 2026

Latent Commonality Expectation-Maximisation for Box-supervised Tree Crown Instance Segmentation

Authors: Thomas Pitts, Kunqi Li, Bin Liang

Organizations: Department of Data Science, University of Technology Sydney

Abstract

Individual tree crown segmentation from aerial imagery underpins tree-level carbon accounting, biodiversity, and restoration monitoring at landscape scale. However, existing models are predominantly trained on dense canopy forest imagery and degrade in savannah and drylands, where tree crowns are sparse, of variable appearance, and underrepresented in annotated benchmarks. These models also typically depend on costly polygon annotations. We introduce LACE (LAtent Commonality Expectation-maximisation), a box-supervised instance segmentation model, evaluated on 0.1 m/px aerial RGB tree crown imagery. LACE uses a frozen DINOv3-web ViT-L/16 encoder, applied at four spatial offsets and interlaced into a denser feature grid, with a lightweight CenterNet-style detection head trained solely on bounding boxes. We use expectation-maximisation to separate recurring appearance, the "treeness", within bounding boxes from surroundings. On the OAM-TCD benchmark test set, LACE reaches a mask AP $_{50}$ of $0.663 \pm 0.001$ (3 seeds) trained on 900 box-annotated images and without mask annotations, above the 0.626 scored by Restor's released mask-supervised Mask R-CNN, which was trained on the full ~4.2k image set. On a sparse-canopy holdout set, mask AP $_{50}$ rises to $0.691$ versus $0.612$ for Detectree2, a mask-supervised baseline. On NeonTreeEvaluation, using the official evaluation code, LACE reaches $0.728 \pm 0.003$ [email protected] (5 seeds) from 23,424 hand-annotated RGB boxes alone, matching the authors' DeepForest model's published 0.719, using under 0.1% of its training annotations and none of its LiDAR-derived 30M-crown pretraining set. By leveraging frozen self-supervised features, LACE matches or surpasses fully-supervised specialist baselines from boxes alone, removing the need for polygon annotation in tree crown instance segmentation for sparse-canopy environments where labelled data is scarce.

Figures & tables

Trained on crown polygons
			GSD	Biome
Method	Output	Input	Extra net(s)	(cm)	Tr	Te	Sv
Mask R-CNN (OAM-TCD) [ 33 ]	Mask	RGB	–	10	$\circ$	$\circ$	$\circ$
Detectree2 [ 18 ]	Mask	RGB	–	10	$\bullet$	$\times$	$\times$
Mask R-CNN / DETR [ 76 ]	Mask	MS+LiD	–	5	$\times$	$\bullet$	$\times$
CrownViM [ 77 ]	Mask	RGB	–	10	$\circ$	$\circ$	$\circ$

Table 1: Representative individual tree crown methods, grouped by the annotation type used for training. Output : boxes or instance masks. Extra net(s) : pretrained networks required at inference beyond the method’s own backbone. Biome coverage refers to evaluation.

Component	Setting
Encoder	DINOv3 ViT-L/16, frozen, layer 24 only, $1024$ dimensions (NEON: layers 21–24, $4096$ )
Feature grid	4 offsets $(\delta_{y},\delta_{x})\in\{0,8\}^{2}$ , interlaced to $8$ px; $256\times 256$ cells per $2048$ px tile
Detector	$1\times 1$ stem to width $256$ ; $5\times$ $3\times 3$ blocks, group norm, ReLU; heatmap and $4$ -channel regression heads; stride $8$
Target encoding	CenterNet Gaussians, grid $256$ , stride $8$ , radius at IoU overlap $0.7$ , floor $\geq 1$ cell
Detector training	Adam, lr $10^{-3}$ , weight decay $10^{-4}$ , cosine schedule, batch $3$ , max $40$ epochs, early stopping on validation box AP 50 (minimum $12$ epochs, patience $2$ ), best-on-validation checkpoint
Decoding	local maxima, score $>0.05$ , top $600$ per tile; operating point $0.4$ for single-threshold metrics

Table 2: Configuration. Encoder and masker-fitting hyperparameters were fixed a priori; the three inference scalars were selected on validation (Section 3.2.4 ); detector optimisation follows standard practice for anchor-free centre-based heads. The ranking rule contributes no fitted quantity (Section 3.2.5 ).

Method	Superv.	Imgs	AP 50	AP 50:95	Box AP 50
Restor Mask R-CNN a	masks	4169	0.626	0.277	0.654
Detectree2 b	masks	900	0.597	0.258	0.591
SelvaBox $\rightarrow$ SAM 3 c	box+SAM	3024	0.569	0.203	0.730
Box2Mask Swin-L d	box	900	0.540	0.198	0.582
Box2Mask R-50 d	box	900	0.387	0.130	0.471
LACE (ours) (3-seed)	box	900	0.663	0.279	0.640

Table 3: Instance segmentation on the OAM-TCD 439-image holdout. Every row was produced and scored by us under one frozen protocol (using standard COCOeval). All are trained on OAM-TCD, and Imgs is the number of OAM-TCD training images seen. Restor and SelvaBox were not fine-tuned by us; the others were.

Method	P	R	$F_{1}$	max R
DeepForest, published a	0.659	0.790	0.719	–
DeepForest, replicated (v2.1.0) b	0.745	0.709	0.726	0.765
Ours (5-seed)	$0.727\pm 0.009$	$0.729\pm 0.009$	$0.728\pm 0.003$	$0.849\pm 0.003$

Table 4: Individual tree-crown detection on the NeonTreeEvaluation benchmark (RGB-only, IoU 0.4, 194 tiles; precision/recall macro-averaged over images). Metrics at each model’s best-

F_{1}

operating point; max R is the recall ceiling (score threshold

\to 0

)

Method	AP 50	AP 75	AP 50:95	Box AP 50
	Canopy neutral
Detectree2	$0.6118$	$0.2155$	$0.2800$	$0.5897$
LACE (ours)	$\mathbf{0.6913}$	$\mathbf{0.2326}$	$\mathbf{0.3148}$	$\mathbf{0.6442}$
	$\pm$ 0.0095	$\pm$ 0.0107	$\pm$ 0.0069	$\pm$ 0.0272

Table 5: Instance segmentation on the sparse-canopy OAM-TCD holdout (

236

tiles,

14{,}937

crowns, no tile shared with the

900

training images). Canopy is scored as iscrowd throughout, as in Table 3 . LACE bands are

\mathrm{ddof}=1

over detector seeds

0

–

2

, Detectree2 is single seed.

Code	Biome (WWF)	Tiles	Crown px	Canopy px
Sparse-canopy slice
7	Trop./Subtrop. Grassland, Savanna & Shrubland	102	2.3%	8.7%
12	Mediterranean Forest, Woodland & Scrub	56	4.8%	7.6%
8	Temperate Grassland, Savanna & Shrubland	31	4.8%	11.5%
13	Desert & Xeric Shrubland	24	6.2%	15.2%
10	Montane Grassland & Shrubland	16	1.1%	1.9%

Table 6: Composition of the sparse-canopy slice. Tiles are selected by WWF biome label, excluding all 900 training tiles. One eligible tile of biome 12 was dropped at build time as unreadable, leaving 236. Crown px and canopy px are the mean per-tile fraction of labelled crown and labelled canopy pixels.

Box-to-mask module	Mean IoU	$\geq 0.5$	$\geq 0.75$	$\geq 0.9$
LACE commonality EM (ours)	$\mathbf{0.770}$	$\mathbf{0.979}$	$\mathbf{0.659}$	$\mathbf{0.051}$
SAM 3, zero-shot a	$0.741$	$0.956$	$0.545$	$0.037$
SAM 3, SelvaMask fine-tuned b	$0.623$	$0.789$	$0.274$	$0.011$

Table 7: Box-to-mask modules prompted with ground-truth boxes, OAM-TCD

439

-tile holdout,

25{,}692

crowns (the

13

crowns whose mask covers fewer than

4

pixels at the

512^{2}

reference raster are excluded).

Feature grid	[email protected]	max Recall	NIWO site F1
Native $16$ px, interpolated to $8$ px	$0.689\pm 0.013$	$0.779\pm 0.009$	$0.427\pm 0.010$
Interlaced $4$ -offset, real $8$ px	$\mathbf{0.728\pm 0.003}$	$\mathbf{0.849\pm 0.003}$	$\mathbf{0.588\pm 0.017}$
$\Delta$	$+0.039$	$+0.070$	$+0.161$

Table 8: Interlaced versus interpolated

8

px features on NeonTreeEvaluation (194 tiles, 5 seeds per arm, detection F1 at IoU

0.4

). Both arms share the same detector, targets and scorer.

Configuration	Mask AP 50	Mask AP 50:95
$\alpha=1.0$ , $\kappa=10$ (3 seeds)	$0.6232\pm 0.0026$	$0.2404\pm 0.0039$
$\alpha=0.3$ , $\kappa=16$ (3 seeds)	$\mathbf{0.6301\pm 0.0054}$	$\mathbf{0.2568\pm 0.0005}$

Table 9: Inference scalars on the OAM-TCD 439-tile test split, selected on 108 held-out validation images. Both rows use detector seeds

0

–

2

, so the comparison is paired.

M-step	E-step	GT Oracle boxes	Predicted boxes
generative ( $\beta=0$ )	absolute	$0.7402$	$0.6717$
generative ( $\beta=0$ )	recentred	$0.7895$	$0.7188$
contrastive ( $\beta=0.5$ )	absolute	$0.7758$	$0.7063$
contrastive ( $\beta=0.5$ )	recentred	$0.7885$	$0.7193$
repulsion alone		$+0.0356$	$+0.0346$
recentring alone		$+0.0493$	$+0.0471$

Table 10: M-step repulsion against E-step recentring,

K_{fg}=16

, EM seed 0, detector seed 0, 108 validation tiles. Mean per-crown mask IoU over 7041 crowns for the oracle-box columns and 5428 box-matched true positives for the predicted-box columns.

Split	Ranking	AP 50	AP 75	AP 50:95
OAM-TCD 439	$s$ (detector score)	$0.630\pm 0.005$	$0.141\pm 0.003$	$0.257\pm 0.001$
OAM-TCD 439	$s^{\prime}=s\,\bar{p}\,m$	$\mathbf{0.663\pm 0.001}$	$\mathbf{0.165\pm 0.008}$	$\mathbf{0.279\pm 0.004}$
Sparse 236	Detectree2 (FT)	$0.612$	$0.216$	$0.280$
	$s$ (detector score)	$0.656\pm 0.012$	$0.197\pm 0.009$	$0.290\pm 0.007$
	$s^{\prime}=s\,\bar{p}\,m$	$\mathbf{0.691\pm 0.010}$	$\mathbf{0.233\pm 0.011}$	$\mathbf{0.315\pm 0.007}$

Table 11: Ranking by the mask posterior. Both splits use the same frozen scorer and identical predictions per seed; only the ordering differs. The sparse split shares no tile with the

900

training images, so it is a generalisation test rather than a second read of the same distribution.

Ranking	AP 50	AP 75	AP 50:95	Val box AP 50
$s$	$0.6250$	$0.1445$	$0.2581$	$0.4982$
$s\,m$ (drop $\bar{p}$ )	$0.6484$	$0.1577$	$0.2719$	$0.5148$
$s\,\bar{p}$ (drop $m$ )	$0.6472$	$0.1638$	$0.2737$	$0.5094$
$s\,\bar{p}\,m$	$\mathbf{0.6615}$	$\mathbf{0.1728}$	$\mathbf{0.2822}$	$\mathbf{0.5177}$

Table 12: Dropping each factor of Equation ( 13 ), detector seed

0

, OAM-TCD

439

(canopy-neutral mask AP) with box AP 50 on the

108

validation images. Every row ranks the same frozen set of

159{,}687

predictions.

Canopy-class fraction	Tiles	Mask AP 50	Box AP 50	Mask $-$ Box
$[0,\,0.10)$	196	$0.5838$	$0.5611$	$+0.0227$
$[0.10,\,0.25)$	95	$0.6900$	$0.6655$	$+0.0245$
$[0.25,\,0.50)$	86	$0.7162$	$0.6902$	$+0.0260$
$[0.50,\,1.00]$	62	$0.6563$	$0.6238$	$+0.0325$

Table 13: Performance by labelled-canopy fraction, 439 OAM-TCD test tiles, 3 seeds. AP is pooled within each bin against that bin’s own ground-truth count. The mask-minus-box gap is the module’s contribution once detection is accounted for.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

	Box IoU	Mask IoU	Mask IoU within box-IoU stratum
Method			$[0.5,0.6)$	$[0.6,0.7)$	$[0.7,0.8)$	$[0.8,0.9)$	$[0.9,1.0]$
Restor Mask R-CNN	$0.781$	$\mathbf{0.766}$	$0.594$	$0.682$	$0.749$	$0.811$	$0.863$
Detectree2 (fine-tuned)	$0.760$	$0.760$	$0.616$	$0.690$	$0.757$	$\mathbf{0.818}$	$\mathbf{0.865}$
SelvaBox $\rightarrow$ SAM 3	$\mathbf{0.798}$	$0.696$	$0.524$	$0.601$	$0.664$	$0.728$	$0.800$
Box2Mask Swin-L	$0.772$	$0.694$	$0.548$	$0.617$	$0.683$	$0.744$	$0.797$
Box2Mask R-50	$0.742$	$0.644$	$0.500$	$0.574$	$0.651$	$0.723$	$0.775$

Appendix

Table 14: Mean per-crown mask IoU within strata of matched box IoU, OAM-TCD

439

-tile holdout. Computed on the

10{,}020

crowns that every row matched.

					Mean crown IoU
Config.	$K_{fg}$ init	$K_{fg}$ final	Eff. rank	Pairwise $\cos$	Oracle	Pred.
$\beta=0$	16	16	$11.43$	$0.234$	$0.7895$	$0.7188$
$\beta=0.1$	16	15	$9.41$	$0.334$	$0.7900$	$0.7194$
$\beta=0.25$	16	13	$6.37$	$0.525$	$0.7901$	$0.7199$
$\beta=0.5$	16	16	$\mathbf{1.66}$	$\mathbf{0.994}$	$0.7885$	$0.7193$
$\beta=0$	2	2	$2.00$	$0.053$	$0.7888$	$0.7181$

Appendix

Table 15: Foreground prototype geometry and accuracy, E-step recentring held on throughout so that

\beta

is the only variable. Accuracy is mean per-crown mask IoU on the 108 validation tiles at EM seed 0 and detector seed 0, matching the geometry columns. Effective rank is

\exp

of the entropy of the normalised singular-value spectrum of the prototype matrix i.e. it counts directions.

Setting	Value
Engine	pycocotools COCOeval , unmodified; iouType segm then bbox
Unit	the $439$ whole $2048^{2}$ holdout tiles; AP pooled over images, not averaged per tile
Categories	one, tree (OAM-TCD cat $=1$ ). With a single category the class mean is degenerate, which is why we write AP 50 rather than mAP 50
IoU thresholds	linspace(0.5, 0.95, 10) , the COCO convention. An earlier arange(0.5, 0.96, 0.05) placed $0.60$ , $0.75$ and $0.85$ one ulp high and rejected IoUs landing exactly there, worth $0.0008$ AP 50:95
Recall thresholds	$101$ -point (COCO default)
Area ranges	all only. Per-instance areas are persisted, so a size-stratified cut can be made later without re-scoring

Appendix

Table 16: Scorer configuration. Settings marked

\dagger

depart from the COCO defaults.

Row	Geometry	Deviations from released configuration
LACE (ours)	whole $2048^{2}$	none; decoder floor $0.05$ and topk $600$ are the protocol values
Restor Mask R-CNN	whole $2048^{2}$ (their own test configuration)	SCORE_THRESH_TEST $0.2\rightarrow 0.05$ ; DETECTIONS_PER_IMAGE $512\rightarrow 600$ ; RPN.PRE/POST_NMS_TOPK_TEST $512\rightarrow 1000$
Detectree2	$1024^{2}$ at $0.5$ overlap, $3\times 3$ subtiles, stitched	INPUT.MIN_SIZE_TEST set to its own MIN_SIZE_TRAIN of $1000$ rather than Detectron2’s inherited COCO default of $800$ , which would downscale our subtiles at test but not at train; stitched at the protocol floor $0.05$ rather than its $0.1$ ; clean_crowns merge (IoU $>0.7$ or containment $>0.85$ )
SelvaBox $\rightarrow$ SAM 3	$1024^{2}$ at $0.5$ overlap, native $0.1$ m/px	their OAM-TCD benchmark configuration, not the $1777$ px at $0.045$ m/px deployment preset, which would resample this imagery $2.22\times$ up; SAM 3 run at the $1777$ px tile size its wrapper defaults to; IoU-NMS $0.7$ merge; their edge-band cull not applied
Box2Mask	as the Detectree2 row	same crops, grid and stitcher; the architecture’s cap of $100$ instances per $1024^{2}$ subtile is left in place

Appendix

Table 17: Per-row prediction geometry. Deviations are from each method’s released configuration.

RPN topk	Dets/tile	Recall	AP 50	AP 50:95
$512$ (released)	$97.9$	$0.6442$	$0.5706$	$0.2575$
$1000$ (Detectron2 default, reported )	$128.6$	$0.7227$	$\mathbf{0.6255}$	$\mathbf{0.2766}$

Appendix

Table 18: Restor’s proposal budget, canopy-neutral mask AP on the

439

tiles. Recall is at IoU

0.5

; the second row is the operating point reported in Table 3 .

Instance Segmentation

Instance segmentation identifies each object and assigns it a separate pixel-level mask in images or video.

See all 157 papers

Highly Detailed and Generalizable Broadleaf Tree Crown Instance Segmentation from UAV Imagery
May 15, 2026Mitsutaka Nakada, Takahiko Ikebata, Kengo Ikebata +7Image Segmentation Remote Sensing Image Segmentation
DeepForest Technologies Co., Ltd · DeepForest Technologies Co., Ltd., Kyoto 600-8006, Japan · YM Lab +9
SegmentAnyTreeV2: Scaling Transformer-Based Tree Instance Segmentation Across Sensors, Platforms, and Forests
Jun 6, 2026Maciej Wielgosz, Stefano Puliti, Rasmus AstrupTransformer Instance Segmentation
Norwegian Institute of Bioeconomy Research (NIBIO), Høgskoleveien 7, 1433 Ås, Norway
SelectAnyTree: A Promptable Instance Segmentation Model for 3D Forest LiDAR Point Clouds
Jun 25, 2026Trung Thanh Nguyen, Daniel Lusk, Kilian Gerberding +103D Point Cloud Segmentation 3D Instance Segmentation
Nagoya University, Japan · University of Freiburg, Germany · University of California, Los Angeles, USA +5

May 15, 2026J/K move · Enter open · S save

Highly Detailed and Generalizable Broadleaf Tree Crown Instance Segmentation from UAV Imagery

Mitsutaka Nakada, Takahiko Ikebata, Kengo Ikebata, Yuji Mizuno+6

DeepForest Technologies Co., Ltd · DeepForest Technologies Co., Ltd., Kyoto 600-8006, Japan · YM Lab+9

Image Segmentation Remote Sensing Image Segmentation Instance Segmentation UAV Remote Sensing

Open paper AskPDF arXiv

We present a highly detailed instance segmentation model for delineating individual tree crowns in natural broadleaf forests using aerial imagery acquired by unmanned aerial vehicles (UAVs). Tree crown delineation in broadleaf forests is more challenging than in other forest types due to diversity of crown shapes and the lack of clearly defined treetops. To address this issue, we developed a deep-learning-based crown segmentation model trained on high-quality annotated crown outlines. We manually delineated 18,507 crown polygons from orthomosaic images collected across seven forests in Japan by skilled annotators, and developed a model based on Mask2Former with multiple backbone architectures. The best model achieved high segmentation performance in structurally complex broadleaf forests using only RGB imagery. This performance was maintained when applied to geographically distinct forests within Japan, as well as to biologically distinct tropical rainforests in Borneo. These results demonstrate that using a large number of high-quality annotated datasets is critical for achieving detailed and generalizable crown segmentation across diverse forest ecosystems. The developed model has been integrated into DF Scanner Pro, a software that supports practical forest monitoring using UAVs, and this implementation is expected to enable a wide range of users to analyze tree-level information in broadleaf forest from UAVs.

Abstract

Instance Segmentation

Explore similar work

Highly Detailed and Generalizable Broadleaf Tree Crown Instance Segmentation from UAV Imagery

SegmentAnyTreeV2: Scaling Transformer-Based Tree Instance Segmentation Across Sensors, Platforms, and Forests

SelectAnyTree: A Promptable Instance Segmentation Model for 3D Forest LiDAR Point Clouds