Are We Recognizing the Jaguar or Its Background? A Diagnostic Framework for Jaguar Re-Identification
Organizations: Hasso Plattner Institute · NVIDIA · Jaguar ID Project · Universidade Federal de Mato Grosso ·
Abstract
Jaguar re-identification (re-ID) from citizen-science imagery can look strong on standard retrieval metrics while still relying on the wrong evidence, such as background context or silhouette shape, instead of the coat pattern that defines identity. We introduce a diagnostic framework for wildlife re-ID with two axes: a leakage-controlled context ratio, background/foreground, computed from inpainted background-only versus foreground-only images, and a laterality diagnostic based on cross-flank retrieval and mirror self-similarity. To make these diagnostics measurable, we curate a Pantanal jaguar benchmark with per-pixel segmentation masks and an identity-balanced evaluation protocol. We then use representative mitigation families, ArcFace fine-tuning, anti-symmetry regularization, and Lorentz hyperbolic embeddings, as case studies under the same evaluation lens. The goal is not only to ask which model ranks best, but also what visual evidence it uses to do so.
Figures & tables
| Rank | Model | mAP (macro) | BG/FG | Mirror Sim | Mirror Tier | Architecture |
|---|---|---|---|---|---|---|
| 1 | MiewID-MSv3 | 0.303 | 0.59 | 0.746 | T1: Aware | EffNetV2-M |
| 2 | DINOv3-ViT CLS | 0.298 | 0.77 | 0.963 | T2: Moderate | ViT-S/16 |
| 3 | MiewID-MSv2 | 0.297 | 0.52 | 0.752 | T1: Aware | EffNetV2-M |
| 4 | HotSpotter+VLAD | 0.288 | 0.54 | 1.000 | T4: Symmetric | SIFT+VLAD |
| 5 | EVA-02 | 0.281 | 0.66 | 0.997 | T4: Symmetric | ViT-L/14 |
| 6 | DINOv3-ViT Pooled | 0.277 | 0.82 | 0.979 | T3: Strong | ViT-S/16 |
| Within-flank | Cross-flank | Cross/Within | Discrim- | |||
| Model | mAP | CMC@1 | mAP | CMC@1 | ratio | inability |
| ArcFace+MiewID (no flip) | 0.716 | 77.5% | 0.477 | 53.9% | 0.666 | 0.417 |
| ArcFace+MiewID (flip aug) | 0.632 | 70.7% | 0.490 | 55.3% | 0.774 | 0.427 |
| (flip no flip) | 0.084 | 6.8pp | +0.013 | +1.4pp | +0.108 | +0.010 |
| Full RGB | Cross-Flank | |||||||
|---|---|---|---|---|---|---|---|---|
| Model | mAP | CMC@1 | CMC@5 | CMC@10 | mAP | CMC@1 | CMC@5 | CMC@10 |
| ArcFace O0 (MSv3) | 0.483 0.011 | 74.3% 0.9% | 80.7% 1.0% | 83.8% 0.4% | 0.576 0.049 | 60.7% 4.9% | 69.0% 4.3% | 75.1% 2.6% |
| ArcFace O1 (MSv3) | 0.469 0.010 | 74.3% 0.9% | 80.4% 1.5% | 82.4% 1.0% | 0.566 0.026 | 62.4% 1.8% | 69.9% 4.7% | 76.2% 5.6% |
| Lorentz O0 (MSv3) | 0.548 0.012 | 76.3% 1.5% | 83.1% 1.0% | 86.3% 0.7% | 0.689 0.032 | 72.0% 4.7% | 81.5% 0.7% | 84.8% 2.5% |
| Lorentz O1 (MSv2) | 0.515 0.014 | 75.0% 1.7% | 83.5% 0.3% | 87.6% 0.9% | 0.653 0.011 | 71.2% 4.9% | 84.4% 1.2% | 88.8% 1.0% |
| Lorentz O1 (MSv3) | 0.557 0.009 | 77.3% 1.7% | 84.3% 1.4% | 87.1% 1.4% | 0.692 0.010 | 72.5% 2.2% | 81.9% 3.4% | 87.0% 2.9% |
Appendix figures & tables29 assets
Supplementary material from the paper’s appendix.
Appendix
| Backbone | Architecture | Pre-training | Dim | Frozen | Fine-tuned |
| Wildlife-specific pre-training | |||||
| MiewID-MSv2 [ 6 ] | EfficientNetV2-M | Wildlife (49 spp.) | 2152 | A, T, L, AS | |
| MiewID-MSv3 [ 6 ] | SwinV2 | Wildlife | 2152 | A, T, L | |
| MegaDescriptor-L [ 4 ] | Swin-L | Wildlife | 1536 | A, T | |
| Self-supervised / multi-task | |||||
| DINOv3-ViT [ 10 ] | ViT-S/16 | Self-sup. (LVD-142M) | 384 | A | |
| Family / model | Training input | Size | Checkpoint selection | Seed handling | Final statistic | Representative checkpoint rule |
|---|---|---|---|---|---|---|
| ArcFace+MiewID | FG cutouts | 440 | Lowest val loss on coreset | Seeds 42/43/44 | Test mean std | Lowest val-loss seed |
| ArcFace+MiewID+AntiSym ( ) | FG cutouts | 440 | Lowest val loss on coreset | Seeds 42/43/44; hyperparams fixed before reruns | Test mean std | Lowest val-loss seed |
| Lorentz+MSv3+O0 | FG cutouts + solidity | 320 | Family: mean val ib-mAP; seed ckpt: val loss | Seeds 42/43/44 | Test mean std | Highest val ib-mAP, lower val loss tiebreak |
| Lorentz+MSv2+O1 | FG cutouts + solidity | 320 | Family: mean val ib-mAP; seed ckpt: val loss | Seeds 42/43/44 | Test mean std | Highest val ib-mAP, lower val loss tiebreak |
| Lorentz+MSv3+O1 | FG cutouts + solidity | 320 | Family: mean val ib-mAP; seed ckpt: val loss | Seeds 42/43/44 | Test mean std | Highest val ib-mAP, lower val loss tiebreak |
| ArcFace+DINOv3 | FG cutouts | 224 | Lowest val loss on coreset | Seeds 42/43/44 | Test mean std | Lowest val-loss seed |
| Family | Seed | Checkpoint | Full-RGB mAP | Full-RGB CMC@1 | Cross-Flank mAP | Cross-Flank CMC@1 | Raw mirror |
|---|---|---|---|---|---|---|---|
| ArcFace O0 (MSv3) | 42 | arcface_miewid_msv3_O0_seed42_best.pth | 0.475 | 73.2% | 0.582 | 65.1% | 0.858 |
| ArcFace O0 (MSv3) | 43 | arcface_miewid_msv3_O0_seed43_best.pth | 0.476 | 75.5% | 0.513 | 53.8% | 0.850 |
| ArcFace O0 (MSv3) | 44 | arcface_miewid_msv3_O0_seed44_best.pth | 0.499 | 74.2% | 0.633 | 63.1% | 0.857 |
| ArcFace O1 (MSv3) | 42 | arcface_miewid_msv3_O1_seed42_best.pth | 0.479 | 75.4% | 0.576 | 64.8% | 0.849 |
| ArcFace O1 (MSv3) | 43 | arcface_miewid_msv3_O1_seed43_best.pth | 0.473 | 74.4% | 0.530 | 60.4% | 0.844 |
| ArcFace O1 (MSv3) | 44 | arcface_miewid_msv3_O1_seed44_best.pth | 0.456 | 73.1% | 0.591 | 61.9% | 0.848 |
| Split | Mean | Median | Min | Max | Frac( ) | |
|---|---|---|---|---|---|---|
| Train | 1,895 | 0.6937 | 0.7094 | 0.2277 | 0.9678 | 0.0770 |
| Test | 371 | 0.6883 | 0.6967 | 0.2388 | 0.9561 | 0.0701 |
| Model | Full mAP | FG mAP | Inpainted mAP | BG/FG | BG+Sil/FG | Risk |
|---|---|---|---|---|---|---|
| Frozen baselines | ||||||
| MiewID-MSv2 | 0.297 | 0.292 | 0.151 | 0.516 | 0.848 | LOW |
| HotSpotter+VLAD | 0.288 | 0.271 | 0.145 | 0.535 | 0.846 | LOW |
| MiewID-MSv3 | 0.303 | 0.304 | 0.178 | 0.586 | 0.869 | LOW |
| EVA-02 | 0.281 | 0.303 | 0.200 | 0.661 | 0.940 | LOW |
| DINOv3-ViT CLS | 0.298 | 0.296 | 0.228 | 0.768 | 1.020 | LOW |
| Metric | Mean | Median | Min | Max | # | # |
|---|---|---|---|---|---|---|
| Legacy BG+Sil/FG | 0.501 | 0.440 | 0.172 | 1.355 | 12 | 7 |
| Inpainted BG/FG | 0.396 | 0.324 | 0.120 | 1.166 | 3 | 2 |
| Silhouette/FG | 0.366 | 0.348 | 0.125 | 0.860 | 0 | 0 |
| Tier | Negative Type | Defeats | Origin |
|---|---|---|---|
| 1 (Hard) | Mirror (LR flip) | Laterality shortcut | Axis 2 |
| 1 (Hard) | Background cutout | Non-coat context memorisation | Axis 1 |
| 2 (Semi-hard) | Closest different individual | Confusable pairs | Offline ranking |
| Backbone | O0 (standard) | O1 (mirror neg) | |
|---|---|---|---|
| MiewID-MSv3 | 13.3% | ||
| MiewID-MSv2 | 6.0% | ||
| ConvNeXt-tiny | 0.6% | ||
| Swin-tiny | 0.7% | ||
| EfficientNetV2 | 1.2% | ||
| MegaDescriptor-L | 1.6% |
| Backbone | Frozen mAP | O0 (ArcFace) | O1 (ArcFace+AntiSym) | BG+Sil/FG (O0) |
|---|---|---|---|---|
| Swin-T | 0.312 | |||
| ConvNeXt-T | 0.326 | |||
| EffNetV2-M | 0.193 | |||
| MegaDesc-L | 0.245 |
| mAP (macro) | CMC@1 (macro) | BG/FG | ||
|---|---|---|---|---|
| baseline | 0.458 | 72.9% | 0.314 | |
| 0.3 | 0.1 | 0.497 | 75.1% | 0.255 |
| 0.3 | 0.3 | 0.480 | 74.0% | 0.244 |
| 0.3 | 0.5 | 0.498 | 74.9% | 0.249 |
| 0.5 | 0.1 | 0.462 | 70.5% | 0.249 |
| 0.5 | 0.3 | 0.480 | 72.2% | 0.244 |
| T0 (Triplet only) | T1 (Triplet + AntiSym) | |||||
|---|---|---|---|---|---|---|
| Backbone | Full mAP | FG mAP | BG/FG | Full mAP | FG mAP | BG/FG |
| MiewID-MSv3 | ||||||
| MiewID-MSv2 | ||||||
| EffNetV2-M | ||||||
| Swin-T | ||||||
| ConvNeXt-T | ||||||
| Model | X-mAP | X-CMC@1 | X-CMC@5 | X-CMC@10 | X/W |
|---|---|---|---|---|---|
| ArcFace O0 (MSv3) | 0.576 0.049 | 60.7% 4.9% | 69.0% 4.3% | 75.1% 2.6% | 0.777 0.064 |
| ArcFace O1 (MSv3) | 0.566 0.026 | 62.4% 1.8% | 69.9% 4.7% | 76.2% 5.6% | 0.780 0.044 |
| Lorentz O0 (MSv3) | 0.689 0.032 | 72.0% 4.7% | 81.5% 0.7% | 84.8% 2.5% | 0.899 0.026 |
| Lorentz O1 (MSv2) | 0.653 0.011 | 71.2% 4.9% | 84.4% 1.2% | 88.8% 1.0% | 0.881 0.028 |
| Lorentz O1 (MSv3) | 0.692 0.010 | 72.5% 2.2% | 81.9% 3.4% | 87.0% 2.9% | 0.876 0.021 |
| Triplet T0 (MSv3) | 0.681 0.013 | 73.0% 2.3% | 81.3% 2.4% | 84.1% 1.2% | 0.923 0.012 |
| Comparison | Supported endpoints | Remaining tested endpoints |
|---|---|---|
| ArcFace O0 vs O1 (MSv3) | axis2 danger margin | All Axis-1 AP endpoints, raw mirror similarity, and cross-flank AP are not supported after Fisher combination and Holm correction. |
| Lorentz O0 vs O1 (MSv3) | none | Full-RGB AP, foreground AP, silhouette AP, inpainted AP, raw mirror similarity, danger margin, and cross-flank AP are all not supported. |
| Triplet T0 vs T1 (MSv3) | inpainted vegetation 1 AP; inpainted vegetation 2 AP; axis2 danger margin; axis2 mirror similarity | Full-RGB AP, foreground AP, silhouette AP, silhouette+bg AP, vegetation 3 AP, and cross-flank AP are not supported. |
| Comparison | Supported endpoints | Note |
|---|---|---|
| ArcFace O0 vs O1 (MSv3) | none | All full-RGB, foreground, inpainted, silhouette, silhouette+bg, and cross-flank CMC@1/5/10 endpoints are not supported. |
| Lorentz O0 vs O1 (MSv3) | none | All full-RGB, foreground, inpainted, silhouette, silhouette+bg, and cross-flank CMC@1/5/10 endpoints are not supported. |
| Triplet T0 vs T1 (MSv3) | none | All full-RGB, foreground, inpainted, silhouette, silhouette+bg, and cross-flank CMC@1/5/10 endpoints are not supported. |
| Method | mAP | BG/FG | BG+Sil/FG | Mirror Sim | Emb. Dim |
|---|---|---|---|---|---|
| HotSpotter+VLAD | 0.288 | 0.535 | 0.845 | 1.000 | 8,192 |
| HotSpotter+Mean | 0.191 | 0.806 | 1.047 | 1.000 | 128 |
| Metric | Frozen DINOv3 | ArcFace+DINOv3 | Change |
|---|---|---|---|
| Test mAP (macro) | 0.298 | 0.458 | 53.7% |
| Test mAP (micro) | 0.270 | 0.479 | 77.4% |
| CMC@1 (macro) | 70.0% | 78.5% | 12.1% |
| CMC@1 (micro) | 72.8% | 82.2% | 12.9% |
| CMC@5 (macro) | 77.6% | 87.6% | 12.9% |
| CMC@5 (micro) | 82.2% | 89.8% | 9.2% |
| Rank | Model | Mirror Sim | Std | BG+Sil/FG | Tier |
|---|---|---|---|---|---|
| 1 | MiewID-MSv3 (frozen) | 0.746 | 0.067 | NA | T1: Laterality-Aware |
| 2 | MiewID-MSv2 (frozen) | 0.752 | 0.064 | 0.850 | T1: Laterality-Aware |
| 3 | ArcFace+MiewID-MSv2 | 0.833 | 0.083 | 0.348 | T1: Laterality-Aware |
| 4 | MegaDescriptor | 0.914 | 0.035 | 1.020 | T2: Moderate |
| 5 | ResNet50 | 0.958 | 0.021 | 1.125 | T2: Moderate |
| 6 | DINOv3-ViT CLS | 0.963 | 0.012 | 1.020 | T2: Moderate |
| Correlation Metric | Value | -value | 95% CI |
|---|---|---|---|
| Spearman | |||
| Kendall |
| Tier | # IDs | % Data | CMC@1 | CMC@5 | CMC@10 |
|---|---|---|---|---|---|
| HEAD (top 33%) | 10 | 65% | 86.2% | 93.1% | 95.4% |
| MEDIUM (middle 33%) | 11 | 26% | 80.0% | 87.0% | 89.0% |
| TAIL (bottom 33%) | 10 | 9% | 70.4% | 81.5% | 85.2% |
| Rank | Model | Regular mAP | Mirror mAP | Ratio | Tier |
|---|---|---|---|---|---|
| 1 | ResNet152 | 0.215 | 0.184 | 0.857 | Pattern-Aware |
| 2 | MiewID-MSv2 | 0.295 | 0.257 | 0.871 | Pattern-Aware |
| 3 | DINOv3-ViT Pooled | 0.278 | 0.243 | 0.874 | Pattern-Aware |
| 4 | I-JEPA | 0.238 | 0.209 | 0.879 | Pattern-Aware |
| 5 | ResNet50 | 0.227 | 0.215 | 0.945 | Moderate |
| 6 | meFeM-B | 0.199 | 0.192 | 0.962 | Lat.-Agnostic |
| Identity | Images | DM | Mean DM | Max DM |
| Overa | 62 | 1 (1.6%) | 0.337 | 0.072 |
| Medrosa | 170 | 1 (0.6%) | 0.378 | 0.053 |
| Katniss | 63 | 0 | 0.291 | 0.000 |
| Apeiara | 20 | 0 | 0.370 | 0.013 |
| Ousado | 179 | 0 | 0.356 | 0.034 |
| All 31 identities | 2 (0.1%) | 0.366 | 0.072 | |
| Model | Raw mirror | Mirror rank pct | Mirror z | Mean danger_margin |
|---|---|---|---|---|
| ArcFace O0 (MSv3) | 0.855 0.004 | 1.000 0.000 | 0.798 0.011 | -0.412 0.006 |
| ArcFace O1 (MSv3) | 0.847 0.002 | 0.667 0.000 | 0.748 0.002 | -0.391 0.004 |
| Lorentz O0 (MSv3) | 0.290 0.002 | 0.333 0.471 | -0.260 1.192 | -0.146 0.003 |
| Lorentz O1 (MSv2) | 0.291 0.002 | 0.333 0.236 | -0.359 0.615 | -0.118 0.001 |
| Lorentz O1 (MSv3) | 0.296 0.008 | 0.833 0.236 | 0.619 0.788 | -0.149 0.008 |
| Triplet T0 (MSv3) | 0.741 0.002 | 0.333 0.000 | 0.124 0.018 | -0.340 0.004 |
| Model | Input | Preprocess | Global descriptor | Dim |
|---|---|---|---|---|
| DINOv3-ViT (CLS) | 224 | HF processor | CLS token | 384 |
| DINOv3-ViT (pooled) | 224 | HF processor | Mean pool patch tokens | 384 |
| DINOv3-ConvNeXt (CLS) | 224 | HF processor | CLS token | 768 |
| DINOv3-ConvNeXt (pooled) | 224 | HF processor | Mean pool patch tokens | 768 |
| MiewID-MSv2 | 440 | Resize + ImageNet norm | Model embedding (GeM + BN) | 2152 |
| MiewID-MSv3 | 440 | Resize + ImageNet norm | Model embedding (GeM + BN) | 2152 |
Animal Re-Identification
Animal re-identification matches observations of the same individual across time, cameras, viewpoints, and changing conditions.
See all 33 papers