CraterBench-R: Instance-Level Crater Retrieval for Planetary Scale
Organizations: Northern Illinois University · University of Arizona
Abstract
Impact craters are a cornerstone of planetary surface analysis. However, while most deep learning pipelines treat craters solely as a detection problem, critical scientific workflows such as catalog deduplication, cross-observation matching, and morphological analog discovery are inherently retrieval tasks. To address this, we formulate crater analysis as an instance-level image retrieval problem and introduce CraterBench-R, a curated benchmark featuring about 25,000 crater identities with multi-scale gallery views and manually verified queries spanning diverse scales and contexts. Our baseline evaluations across various architectures reveal that self-supervised Vision Transformers (ViTs), particularly those with in-domain pretraining, dominate the task, outperforming generic models with significantly more parameters. Furthermore, we demonstrate that retaining multiple ViT patch tokens for late-interaction matching dramatically improves accuracy over standard single-vector pooling. However, storing all tokens per image is operationally inefficient at a planetary scale. To close this efficiency gap, we propose instance-token aggregation, a scalable, training-free method that selects K seed tokens, assigns the remaining tokens to these seeds via cosine similarity, and aggregates each cluster into a single representative token. This approach yields substantial gains: at K=16, aggregation improves mAP by 17.9 points over raw token selection, and at K=64, it matches the accuracy of using all 196 tokens with significantly less storage. Finally, we demonstrate that a practical two-stage pipeline, with single-vector shortlisting followed by instance-token reranking, recovers 89-94% of the full late-interaction accuracy while searching only a small candidate set. The benchmark is publicly available at hf.co/datasets/jfang/CraterBench-R.
Figures & tables
| Property | Value |
|---|---|
| Crater identities | 25,000 |
| Gallery images | 50,000 (2 and 3 context) |
| Query identities | 1,000 |
| Query images | 5,000 (5 views each) |
| Regions | 8 MC quadrangles |
| Diameter median | 1.5 km |
| Model | Params | Pool | R@1 | R@5 | R@10 | mAP |
|---|---|---|---|---|---|---|
| CNN (ImageNet supervised) | ||||||
| EfficientNet-B0 | 4 M | GAP | .150 | .214 | .250 | .248 |
| ResNet-50 | 24 M | GeM | .142 | .217 | .248 | .244 |
| Self-supervised ViT | ||||||
| ViT-S/16 DINO | 22 M | CLS | .273 | .360 | .402 | .420 |
| ViT-B/8 DINO | 86 M | GeM | .304 | .379 | .409 | .461 |
| DINO | MarsDINO | |||||
| Method | Ret. | B/img | R@1 | mAP | R@1 | mAP |
| Single vector (1,536 B) | ||||||
| Best pool (Tab. 2 ) | SV | 1,536 | .273 | .420 | .269 | .412 |
| NetVLAD+PCA | SV | 1,536 | .286 | .437 | .269 | .413 |
| Late interaction, (24 KB) | ||||||
| VLAD (global) | LI | 24,576 | .307 | .459 | .298 | .452 |
| R@1 | mAP | R@ | Time (ms) | ||
| ViT-S/16 DINO (CLS shortlist, FPS/hard seeds) | |||||
| Stage 1 only | — | .273 | .422 | — | 0.6 |
| +Rerank | 50 | .446 | .614 | .497 | 2.0 |
| +Rerank | 100 | .463 | .644 | .534 | 3.6 |
| +Rerank | 200 | .478 | .668 | .576 | 6.4 |
| +Rerank | 500 | .497 | .695 | .629 | 15.0 |
| Model | Pretraining | Params | Pool | R@1 | R@5 | R@10 | mAP |
|---|---|---|---|---|---|---|---|
| CNNs (ImageNet-1k supervised) | |||||||
| ResNet-18 | Sup. IN-1k | 11 M | GAP | .102 | .155 | .182 | .179 |
| ResNet-50 | Sup. IN-1k | 24 M | GeM | .142 | .217 | .248 | .244 |
| ResNet-101 | Sup. IN-1k | 43 M | GAP | .103 | .170 | .200 | .184 |
| ConvNeXt-T | Sup. IN-1k | 28 M | GeM | .075 | .121 | .146 | .134 |
| EfficientNet-B0 | Sup. IN-1k | 4 M | GAP | .150 | .214 | .250 | .248 |
| CLS | Mean | Max | GeM ( ) | |||||
| Backbone | R@1 | mAP | R@1 | mAP | R@1 | mAP | R@1 | mAP |
| DINO v1 | ||||||||
| ViT-S/16 DINO | .273 | .420 | .177 | .279 | .126 | .210 | .258 | .392 |
| ViT-S/8 DINO | .296 | .454 | .162 | .257 | .110 | .185 | .185 | .286 |
| ViT-B/16 DINO | .295 | .454 | .226 | .351 | .194 | .313 | .262 | .399 |
| ViT-B/8 DINO | .303 | .465 | .263 | .402 | .226 | .356 | .304 | .461 |
| Model | Pool | R@1 | R@5 | R@10 | mAP |
|---|---|---|---|---|---|
| ResNet-18 | GAP | .102 | .155 | .182 | .179 |
| ResNet-50 | GAP | .103 | .159 | .191 | .182 |
| ResNet-50 | GeM | .142 | .217 | .248 | .244 |
| ResNet-101 | GAP | .103 | .170 | .200 | .184 |
| ConvNeXt-T | GAP | .072 | .120 | .143 | .130 |
| ConvNeXt-T | GeM | .075 | .121 | .146 | .134 |
| DINO | MarsDINO | |||
|---|---|---|---|---|
| Strategy | R@1 | mAP | R@1 | mAP |
| Attention | .507 | .716 | .453 | .642 |
| Norm Attn | .507 | .717 | .450 | .641 |
| Norm | .505 | .714 | .374 | .549 |
| Random | .477 | .681 | .395 | .576 |
| Spatial grid | .457 | .659 | .390 | .566 |
| hard_top1 | soft_top2 | soft_top4 | group_dense | Raw | |||||
|---|---|---|---|---|---|---|---|---|---|
| R@1 | mAP | R@1 | mAP | R@1 | mAP | R@1 | mAP | mAP | |
| 4 | .108 | .180 | .104 | .174 | .100 | .168 | .109 | .181 | .168 |
| 8 | .231 | .378 | .219 | .358 | .189 | .304 | .192 | .303 | — |
| 16 | .425 | .623 | .416 | .612 | .394 | .582 | .296 | .457 | .444 |
| 32 | .514 | .726 | .511 | .724 | .497 | .708 | .432 | .628 | — |
| 64 | .541 | .759 | .543 | .760 | .536 | .753 | .492 | .698 | .716 |
| hard_top1 | soft_top2 | soft_top4 | gtp_dense | Raw | |||||
|---|---|---|---|---|---|---|---|---|---|
| R@1 | mAP | R@1 | mAP | R@1 | mAP | R@1 | mAP | mAP | |
| 4 | .163 | .267 | .166 | .270 | .169 | .275 | .171 | .276 | .167 |
| 8 | .258 | .400 | .257 | .399 | .255 | .394 | .251 | .388 | — |
| 16 | .351 | .528 | .343 | .517 | .341 | .508 | .322 | .484 | .446 |
| 32 | .415 | .602 | .407 | .591 | .395 | .577 | .366 | .536 | — |
| 64 | .457 | .650 | .449 | .639 | .439 | .626 | .388 | .561 | .642 |
| DINO | MarsDINO | |||||||
|---|---|---|---|---|---|---|---|---|
| Attention | FPS | Attention | FPS | |||||
| R@1 | mAP | R@1 | mAP | R@1 | mAP | R@1 | mAP | |
| 4 | .109 | .181 | .084 | .152 | .171 | .276 | .113 | .201 |
| 8 | .203 | .326 | .231 | .378 | .258 | .400 | .227 | .371 |
| 16 | .336 | .504 | .425 | .623 | .353 | .524 | .351 | .528 |
| 32 | .449 | .647 | .514 | .726 | .415 | .602 | .414 | .598 |
| Method | B/tok | Comp. | DINO | MarsDINO |
|---|---|---|---|---|
| FP32 | 1536 | 1 | .726 | .602 |
| FP16 | 768 | 2 | .726 | .602 |
| INT8 | 388 | 4 | .726 | .602 |
| PQ ( ) | 96 | 16 | .715 | .590 |
| PQ ( ) | 48 | 32 | .664 | .529 |
| DINO | MarsDINO | |||||
|---|---|---|---|---|---|---|
| Late | Mean | Max | Late | Mean | Max | |
| 4 | .181 | .144 | .107 | .276 | .281 | .245 |
| 8 | .326 | .207 | .141 | .400 | .343 | .311 |
| 16 | .504 | .267 | .193 | .524 | .376 | .362 |
| 32 | .647 | .323 | .239 | .602 | .391 | .390 |
| 64 | .734 | .322 | .268 | .650 | .401 | .404 |
Image Retrieval
Image retrieval finds and ranks images from visual, textual, or combined queries across large and specialized collections.
See all 85 papers