Beyond Family Labels: A Taxonomic Ornstein-Uhlenbeck Prior for Avian 3D Shape Recovery
Authors: Yunchan Jeon, SooGon Kim, Seungwook Kim, CheolWon Lee, Jongmin Lee
Organizations: Department of Computer Science and Engineering, Chung-Ang University, South Korea · GenGenAI, South Korea · Department of Computer Engineering, Konkuk University, South Korea
Recovering 3D pose and shape of birds from single images is bottlenecked by the long tail of avian diversity: out of ~10,000 species, only a few dozen have 3D-labeled training data, and in-the-wild test sets routinely contain species never seen at training. Generalizing across clades is therefore a central problem. The current state of the art, AniMer+, mitigates it with a family-level contrastive loss that conveys only a binary same-family indicator per pair and discards the continuous structure of the species tree. We propose a taxonomic Ornstein-Uhlenbeck prior on the shape code of the AVES parametric bird mesh model: a multivariate Gaussian whose covariance decays exponentially with Linnaean rank distance, with a learnable selection strength and noise scale. The prior generalizes family contrastive into a continuous, generative form, exposing the graded cross- and within-family similarity that a binary indicator cannot, and brings continuous comparative phylogenetics in as a biology-grounded inductive bias for the long tail. Our method improves over AniMer+ on both metrics of the unseen-species CowBird benchmark and on CtrlAVES3D, while remaining comparable on seen-species CUB. A permutation ablation isolates taxonomic structure from generic Gaussian smoothing, and a rank-granularity ablation identifies cross-order distinctions (the structure beyond a family-vs-not indicator) as the dominant source of the unseen-species gain.
Figures & tables
Figure 1: Beyond family labels. The 17 training species of CtrlAVES3D on the Linnaean tree [ 2 ] , colored by order. The height at which two lineages join marks their lowest shared rank, from same family up to outgroup; each rank sets the distance dtax that our prior reads ( Figure 2 b). The family-supervised contrastive loss of AniMer+ [ 1 ] keeps a single bit of this tree: whether a pair joins below the dashed red line. Our Ornstein–Uhlenbeck (OU) prior uses every rank, pulling the shape codes of two species together more strongly the more closely they are related, so the shape space carries graded structure rather than a same-family indicator.
Figure 2: Method at a glance. (a) The AniMer+ [ 1 ] encoder produces AVES [ 12 ] shape codes β from a single image. (b) Across species, β is regularized by an OU [ 10 ] prior parameterized by learnable (α,σ,μ) over a fixed Linnaean [ 2 ] rank distance matrix; the NLL of this Gaussian is LOU . (c) LOU replaces the family-level contrastive term of AniMer+ ( wcon=0 ), so that on birds Ltotal=LAVES+λOULOU . Blue: the existing network and fixed taxonomy. Orange: what we add.
CUB
CowBird
CtrlAVES3D
Method
AUC ↑
PCK ↑
AUC ↑
PCK ↑
AUC ↑
AVES [ 12 ]
79.70
25.20
46.10
5.60
85.90
HMR2.0 [ 33 ]
87.70
50.00
59.10
12.20
91.60
AniMer-A [ 20 ]
91.90
74.80
65.70
19.40
92.90
AniMer+ [ 1 ]
89.50
59.40
59.30
16.40
93.00
Ours
89.30±0.02
57.84±0.09
60.80±0.10
18.86±0.27
93.05±0.01
Table 1: Comparison against AniMer+ [ 1 ] on the avian benchmarks. CUB [ 32 ] and the CtrlAVES3D [ 29 ] test split evaluate seen -species; CowBird [ 6 ] is the unseen -species out-of-distribution (OOD) benchmark. For ours we report the mean ± sample standard deviation over five random seeds. AniMer-A [ 20 ] is the avian-only variant; AniMer+ and ours are unified bird+mammal models; AVES and HMR2.0 are the baselines reported by AniMer+.
Figure 3: Real vs. permuted taxonomy ablation. CowBird [ 6 ] AUC and [email protected] under three configurations: (a) the AniMer+ [ 1 ] baseline, (b) the OU prior with a label-permuted distance matrix, and (c) the OU prior with the real Linnaean [ 2 ] distance matrix (both OU runs at λOU=10−4 , constant learning rate; Section 4.1 ).
CUB [ 32 ]
CowBird [ 6 ]
CtrlAVES3D [ 29 ]
Dtax variant
AUC ↑
PCK ↑
AUC ↑
PCK ↑
AUC ↑
permuted (null)
88.87
54.90
60.65
18.49
92.95
family-only
89.09
56.35
60.84
19.09
93.06
order-only
89.10
56.42
60.81
19.20
93.07
Linnaean (full, ours)
88.92
55.16
61.32
20.25
92.99
Table 2: Rank-granularity ablation : progressively richer distance matrices on the same operating point as Figure 3 . The family-only kernel uses values in {0,1,5} (same family vs. not), the order-only refinement uses {0,1,2,5} (adds the same-order distinction), and the full Linnaean kernel uses {0,1,2,4,5,6} (cross-order ordinals as the default). All three kernels use 0 for the same species and 1 for the same family, and differ only above the family level; genus-level structure cannot be varied on this training set, because the 17 species contain no congeneric pair. We use 5 rather than 6 as the cross-order placeholder for the family-only and order-only kernels so that they share a single cross-order bin, isolating the rank-granularity axis from changes in cross-order distance scale.
Variant
pa-MPJPE ↓
pa-MPVPE ↓
AniMer+ [ 1 ]
655.61
708.97
AniMer+ w/ OU (ours)
649.89
699.01
Table 3: Procrustes-aligned 3D errors on the CtrlAVES3D test split ( 309 images, 17 species) . pa-MPJPE denotes the mean per-joint position error; pa-MPVPE denotes mean per-vertex position error. Lower is better. Ours is a single run (seed 0 ). Values follow our canonical-frame scaling; see Appendix S6 for details.
Figure 4: The prior tightens the shape codes without distorting phylogenetic signal. (a, b) What tightens: k -NN family purity rises and the spread of the per-species mean β falls with λOU on CtrlAVES3D [ 29 ] training codes; the 10−4 bars are a single constant-LR run. (c) What is preserved: the β distance between the mean shape codes of two species measures how different the model predicts their shapes to be. All 136 species pairs, sorted along the x -axis by this distance without the prior (gray), move closer at λOU=10−5 (orange, seed 0 ) but stay in nearly the same order.
Figure 5: Qualitative comparison on two CUB [ 32 ] test images (top) and two CowBird [ 6 ] test instances (bottom) . For each species, the first panel shows the input with the ground-truth keypoints (green dots) and a red circle on the baseline’s worst keypoint; the retrained AniMer+ baseline (red crosses) and the OU fine-tune (blue crosses) follow, and gray lines join each prediction to its ground-truth keypoint. The badge is the per-image [email protected].
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
Variant
Linnaean [ 2 ]
Family
Order
Jetz [ 30 ] (My)
Jetz norm
Perm ( s42 )
Perm ( s101 )
Perm ( s202 )
AniMer+ [ 1 ] retrain @30K (no OU)
+0.106
+0.119
+0.166
+0.358
+0.358
+0.138
−0.152
−0.201
Cosine LR (no OU)
+0.105
+0.116
+0.167
+0.354
+0.354
+0.136
−0.154
−0.202
Constant LR + OU λ=10−5
+0.070
+0.121
+0.136
+0.354
+0.354
+0.129
−0.164
−0.189
Cosine LR + OU λ=10−5 ( ours )
+0.068
+0.119
+0.135
+0.346
+0.346
+0.123
−0.170
−0.189
Stronger OU regime ( λ=10−4 , single seed)
λ=10−4 + real dtax
+0.030
+0.129
+0.094
+0.330
+0.330
+0.080
−0.174
−0.154
Appendix
Table S1: Spearman correlation ρ between β -distance and phylogenetic distance. Per-species means computed from inference on the 6464-image CtrlAVES3D [ 29 ] train split, N=17 species, D=15 shape dims; correlated against various phylogenetic distance matrices across six variants. Bold = the strongest non-permuted phylogeny within each row. The Jetz [ 30 ] patristic distance dominates the rank in every variant.
Variant
Kmult ( p )
Mantel r ( p )
σβˉ
AniMer+ [ 1 ] released checkpoint
0.583 ( .026 )
+0.360 ( .008 )
0.169
Cosine LR (no OU)
0.581 ( .034 )
+0.358 ( .006 )
0.170
Cosine LR + OU λ=10−5 ( ours , seed 0)
0.577 ( .037 )
+0.351 ( .007 )
0.144
Cosine LR + OU λ=10−5 , permuted dtax
0.577 ( .036 )
+0.351 ( .016 )
0.144
Constant LR + OU λ=10−4 , real dtax
0.618 ( .023 )
+0.317 ( .032 )
0.105
Constant LR + OU λ=10−4 , permuted dtax
0.627 ( .019 )
+0.328 ( .028 )
0.105
Appendix
Table S2: Phylogenetic signal of the per-species mean shape codes on the Jetz [ 30 ] tree. Kmult : multivariate Blomberg’s K [ 28 , 37 ] ( 1 = Brownian expectation, 0 = no phylogenetic signal); Mantel r : correlation between pairwise β distance and patristic distance over the 136 species pairs; σβˉ : mean per-dimension s.d. of the per-species means. Permutation p in parentheses ( 999 label shuffles).
Variant
k -NN purity (family)
k -NN purity (order)
Silhouette (family)
AniMer+ [ 1 ] retrain @30K (no OU)
0.621
0.714
−0.043
Cosine LR (no OU)
0.610
0.705
−0.042
Cosine LR + OU λ=10−5
0.678
0.755
−0.034
λ=10−4 + real dtax
0.725
0.793
−0.018
Appendix
Table S3: β -space clustering quality by Linnaean [ 2 ] rank label. Higher k -NN purity = β -codes of same family/order are locally more clumped. Stronger OU [ 10 ] regimes monotonically increase purity.
i
Species (binomial)
Family
Order
0
Phoebastria immutabilis
Diomedeidae
Procellariiformes
1
Passerina ciris
Cardinalidae
Passeriformes
2
Cardinalis cardinalis
Cardinalidae
Passeriformes
3
Corvus brachyrhynchos
Corvidae
Passeriformes
4
Colaptes auratus
Picidae
Piciformes
5
Tyrannus forficatus
Tyrannidae
Passeriformes
Appendix
Table S4: The 17 CtrlAVES3D species in row-index order in Dtax . Eight species belong to Passeriformes; the remaining nine span six non-Passeriformes orders. Species set: CtrlAVES3D [ 29 ] ; binomial nomenclature, family, and order classification follow the IOC World Bird List [ 2 ] .
Figure S1: The full 17×17 taxonomic rank distance matrix Dtax used by the OU [ 10 ] prior. Diagonal entries are 0 (same species); off-diagonal entries take values {1,2,4,5,6} , representing same family, same order, sister-order, cross-order, and outgroup respectively. The species ordering on the axes matches Table S4 ; the dark first row/column reflect the single-species Procellariiformes outgroup ( Phoebastria immutabilis , i=0 ).
dtax
Count
Pairs
Interpretation
1
8
4 unordered
Same family
2
54
27 unordered
Same order, different family
4
8
4 unordered
Sister-orders within Telluraves
5
170
85 unordered
Cross-order, non-sister
6
32
16 unordered
Outgroup (Procellariiformes vs. all)
total off-diag
272
136 unordered
Appendix
Table S5: Off-diagonal distribution of Dtax . Counts are over the 17×16=272 ordered off-diagonal pairs; an unordered species pair contributes 2 entries (one in each direction). Diagonal entries (same species, d=0 ) are not included.
Table S6: Per-trial results at the headline operating point. Full hyperparameter setup in Appendix S9 . Each row is one independent rerun (training seed in parentheses); Table 1 reports the Mean ± s.d. row. CUB is scored on the official test split.
Figure S2: Evaluation metrics across the swept OU [ 10 ] regularization strength λOU . Solid lines are [email protected] on the seen-species CUB [ 32 ] and the unseen-species CowBird [ 6 ] ; dashed lines are AUC. Dotted lines are the AniMer+ paper values; the shaded region and stars mark λOU=10−5 , the value we adopt. All points are constant-LR runs. The sweep is scored on the development list of Appendix S3 . The figure illustrates the seen-vs-unseen trade-off: seen-species accuracy regresses smoothly with λOU while unseen-species accuracy peaks at intermediate values.
Table S7: Real vs. permuted Dtax on the 183 CUB species outside the fine-tuning taxonomy (official test split; same checkpoints as Figure 3 ). Δ = real − permuted in percentage points; 95% intervals from a species-clustered bootstrap; “real > perm.” counts species whose mean is strictly higher under the real matrix.
Table S8: Effect of OU-LR multiplier k on eval at λOU=10−4 .
Figure S3: OU [ 10 ] hyperparameter learning trajectories across λOU values and real vs. permuted taxonomies. All runs train for 30K steps with k=100 . Both scalars start at 1 , marked by the dashed line. α rises and σ falls in every run, and the change grows with λOU . The λOU=10−5 run uses a cosine LR and the others a constant LR. The real and permuted taxonomy variants give nearly identical trajectories, showing that the motion of α and σ is driven by the OU kernel shape rather than tree content .
Figure S4: α and σ over 30K steps for the real and permuted Dtax at λOU=10−4 . All runs use a constant LR, and each taxonomy has three runs with seeds 0 , 42 and 123 . Both scalars start at 1 , marked by the dashed line. The real and permuted curves stay close throughout training, and the gap between them is on average smaller than the spread across seeds. Section 4.3 gives the complementary final-eval evidence.
Variant
pa-MPJPE [ 31 ] ↓
pa-MPVPE ↓
AniMer+ [ 1 ] pretrained checkpoint
655.61
708.97
AniMer+ [ 1 ] + OU (ours)
649.89
699.01
λ=10−4 + real dtax
650.91
697.98
λ=10−4 + perm dtax
652.15
699.31
λ=5⋅10−4
653.74
704.56
λ=10−3
651.20
701.56
Appendix
Table S9: Procrustes-aligned [ 31 ] 3D errors on the CtrlAVES3D [ 29 ] test split (309 images, 17 species). pa-MPJPE = mean per-joint position error after Procrustes alignment; pa-MPVPE = mean per-vertex position error. Both in canonical AVES template units (see Caveat). Ours is the seed- 0 run (Trial 1 of Table S6 ).
Figure S5: Eight CowBird [ 6 ] test cases with the largest per-image mean-error reduction under the OU [ 10 ] fine-tune. The convention follows Figure 5 , with a first panel showing the input and the ground-truth keypoints in green. The retrained AniMer+ [ 1 ] baseline has red crosses and ours has blue crosses. Gray lines join each prediction to its ground-truth keypoint, and the red circle marks the keypoint the baseline misplaces most. The badge is the per-image mean keypoint error over the visible keypoints in normalized crop units, so lower is better. Best viewed in electronic format.
Bbox-area tercile
n
Mean baseline max-err
Mean ours max-err
Δ (improvement rate)
Small (lowest tercile)
38
0.435
0.396
+0.038 ( 57.9% improved)
Medium (mid tercile)
38
0.503
0.482
+0.021 ( 52.6% improved)
Large (highest tercile)
38
0.620
0.628
−0.007 ( 55.3% improved)
Appendix
Table S10: CowBird [ 6 ] per-image Δ max-err (baseline − ours) split by bbox area tercile. Small birds (small image footprint) receive the largest OU [ 10 ] benefit, medium birds a smaller one, and large birds a slightly negative change on average. Max-err is the largest keypoint error in an image.
Figure S6: Per-image Δ max-err on CowBird [ 6 ] by relative bbox area. The 114 test images are split into three terciles of 38 images each. Δ is baseline minus ours, so a positive bar means ours is better, and the label under each bar gives the share of images improved. The improvement is largest on small birds at +0.038 , smaller on medium birds at +0.021 , and slightly negative on large birds at −0.007 .
Table S11: Paired per-image comparison on CowBird [ 6 ] ( 114 instances; seed- 0 headline run vs. the released AniMer+ [ 1 ] checkpoint, every image weighted equally).
Figure S7: Eight further CUB [ 32 ] test cases. The convention follows Figure 5 , with a first panel showing the input and the ground-truth keypoints in green. The retrained AniMer+ [ 1 ] baseline has red crosses and ours has blue crosses. The badge is the per-image [email protected], so higher is better. Best viewed in electronic format.
Variant
ms @ bs = 1
ms @ bs = 8
Peak mem @ bs = 8 (MB)
Params (M)
AniMer+ [ 1 ] pretrained
19.0
74.0
3160.9
785.67
AniMer+ retrain @30K
18.9
73.9
3160.9
785.67
Ours
18.9
74.0
3160.9
785.67
Appendix
Table S12: Inference-time cost across the three checkpoints. Measured on a single RTX 4090 with fp32 forward passes. Latency is the median over 20 timed iterations after 3 warm-up steps; peak memory and parameter counts are deterministic. The architecture is unchanged across the three rows, so parameter counts coincide exactly.