NEO-BENCH: A New Multi-Source Benchmark for Generalizable Astronomical Streak Detection
Organizations: Interlake High School Bellevue, WA, USA
Abstract
Near-Earth Objects (NEOs) can appear as faint streaks in long-exposure astronomical images. Detecting these streaks across diverse observatories requires methods that remain reliable despite differences in image quality, orientation, sky background, and noise. However, existing detectors are commonly evaluated using data from only one source, providing limited evidence of cross-source generalization. We introduce NEO-Bench, a multi-source benchmark containing 8,376 images from five astronomical-image datasets. The sources include the Hubble Space Telescope, a Stellina smart telescope, the United Arab Emirates Meteor Monitoring Network, a TETRA1 telescope using a Celestron C14 with Fastar, and the Roboflow Asteroid dataset. We converted the data to a common YOLO format, audited a sample of labels, and defined within-source and leave-one-source-out evaluation protocols. We evaluated four approaches: Hough, Radon, Gaussian PSF, and YOLO26L. Leave-one-source-out F1 decreased in 14 of 20 image-level method-source pairs and 13 of 20 [email protected] localization pairs. Across the datasets categorized as medium or hard, F1 decreased in 11 of 12 image-level pairs and 9 of 12 localization pairs. These results show that cross-source performance remains inconsistent and that reliable generalization across astronomical imaging sources remains an open challenge. The benchmark, code, and data are publicly available.
Figures & tables
| Dataset | Difficulty | Train | Val. | Test | Total | Streak-positive | Streak-negative | Boxes | Box/label processing |
|---|---|---|---|---|---|---|---|---|---|
| StreakMind | Hard, background artifacts | 1,484 | 278 | 94 | 1,856 | 840 | 1,016 | 858 | OBB polygons converted to AABBs |
| UAEMMN | Hard, faint streaks | 1,103 | 139 | 140 | 1,382 | 740 | 642 | 740 | Claude Opus 4.6 AABBs |
| StreaksYoloDataset | Medium, bright streaks | 1,722 | 333 | 333 | 2,388 | 1,642 | 746 | 1,642 | Original YOLO AABBs |
| Hubble Asteroid Hunter | Easy, no negatives, artifacts | 1,361 | 170 | 170 | 1,701 | 1,701 | 0 | 1,701 | WCS-projected endpoints; 5-pixel padding |
| Roboflow | Easy, very few negatives | 929 | 80 | 40 | 1,049 | 1,046 | 3 | 2,174 | Original YOLO AABBs |
| Combined | – | 6,599 | 1,000 | 777 | 8,376 | 5,969 | 2,407 | 7,115 | – |
| Dataset | Method | Within-source (in-domain) | Leave-one-out (out-of-domain) | |||||
|---|---|---|---|---|---|---|---|---|
| P | R | P | R | |||||
| StreakMind | YOLO26L | 1.000 | 0.976 | 0.988 | 0.923 | 0.857 | 0.889 | |
| Hough | 0.756 | 0.738 | 0.747 | 0.447 | 1.000 | 0.618 | ||
| Radon | 0.607 | 0.810 | 0.694 | 0.577 | 0.976 | 0.726 | ||
| Gaussian PSF | 0.589 | 0.786 | 0.673 | 0.506 | 1.000 | 0.672 | ||
| UAEMMN | YOLO26L | 0.961 | 1.000 | 0.980 | 0.800 | 0.919 | 0.855 | |
Cross-Domain Generalization
Cross-domain generalization studies how machine-learning models retain reliable performance when data, sensors, populations, or tasks differ from training.
See all 181 papers