Oriented Semi-DETR for Semi-Supervised Aerial Object Detection
Organizations: University of Chinese Academy of Sciences · Peng Cheng Laboratory
Abstract
Semi-supervised object detection has recently been extended to DETR, where oriented boxes introduce an ambiguity that does not arise for axis-aligned boxes: the map from parameters to rectangles is two-to-one, since and denote the same rotated rectangle. Losses and matching costs are computed from the parameters, so two boxes that coincide in space can be far apart in the metric that drives training. A correctly localized student box is then penalized by the regression loss, and the inflated matching cost leads the Gaussian mixture model (GMM) filter to discard it. We propose Oriented Semi-DETR, which removes this ambiguity wherever a pseudo box enters training. Swap-aligned Target Selection regresses towards the encoding closer to the student prediction. A Swap-invariant Reliability Cost minimizes over the two encodings inside the matching cost, so the filter decides on geometry alone. Swap-aware Pseudo-label Denoising starts denoising earlier, ramps its loss weight linearly, and weights each query by the teacher score and encoding margin. All three are parameter-free and require no additional forward pass. Across all label ratios of DOTA-v2.0, Oriented Semi-DETR outperforms DETR-based supervised and semi-supervised baselines, with the largest gains at the strict IoU threshold.
Figures & tables
| Method | 10% labeled | 20% labeled | 30% labeled | Fully labeled | ||||||||
| mAP | mAP | mAP | mAP | |||||||||
| DINO [ 21 ] | 40.72 | 21.76 | 22.49 | 46.31 | 24.94 | 25.82 | 48.11 | 26.71 | 27.16 | 49.11 | 26.91 | 27.51 |
| Semi-DETR [ 22 ] | 43.94 | 25.62 | 25.82 | 48.01 | 28.10 | 28.10 | 50.63 | 29.47 | 29.60 | 51.11 | 31.50 | 30.81 |
| Sparse Semi-DETR [ 9 ] | 44.16 | 25.49 | 25.83 | 48.38 | 28.28 | 28.51 | 50.24 | 29.40 | 29.53 | 51.71 | 32.26 | 31.22 |
| Oriented Semi-DETR | 45.92 | 27.77 | 27.61 | 49.89 | 31.67 | 30.69 | 52.11 | 33.51 | 32.19 | 53.16 | 35.07 | 33.23 |
| Pseudo-label DN setting | 10% labeled | 20% labeled | 30% labeled | Fully labeled | ||||||||
| mAP | mAP | mAP | mAP | |||||||||
| Early pseudo-label DN ramp | 45.56 | 27.88 | 27.52 | 49.19 | 32.15 | 30.69 | 51.84 | 33.57 | 31.96 | 52.79 | 34.86 | 33.15 |
| Early ramp + teacher score | 45.92 | 27.77 | 27.65 | 49.64 | 31.95 | 30.50 | 52.08 | 33.60 | 32.18 | 53.02 | 34.50 | 32.99 |
| Early ramp + teacher score + encoding margin | 45.92 | 27.77 | 27.61 | 49.89 | 31.67 | 30.69 | 52.11 | 33.51 | 32.19 | 53.16 | 35.07 | 33.23 |
| Regr. target | Sup. DN | Unsup. DN | Pseudo-DN schedule | mAP | ||
| – | 49.15 | 28.68 | 28.69 | |||
| ✓ | – | 50.47 | 30.55 | 30.14 | ||
| ✓ | ✓ | – | 51.44 | 33.54 | 31.89 | |
| ✓ | ✓ | ✓ | – | 51.98 | 33.40 | 31.95 |
| ✓ | ✓ | ✓ | early ramp | 51.84 | 33.57 | 31.96 |
| ✓ | ✓ | ✓ | early ramp | 52.08 | 33.60 | 32.18 |
| mAP | |||
| (cosine) | 49.83 | 31.59 | 30.59 |
| (raw L1) | 51.53 | 33.04 | 31.76 |
| 51.98 | 33.40 | 31.95 |
| Train step | Matched pairs | Sigma | Assign. flip | Filter flip |
| 20k | 4671 | 16.89% | 18.88% | 7.02% |
| 180k | 3826 | 3.89% | 4.89% | 2.88% |
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
| periodic | smooth | |||
| cosine | 1.00 | 2.00 | yes | yes |
| wrapped L1 | 0.25 | 0.50 | yes | no |
| raw L1 | 0.25 | 0.50 | no | no |
| Term | Fixed cost |
| , | |
| pixels | |||
| input | |||
| correct swap | |||
| normalized swap |
| Configuration | Reg. | Sup. DN | Unsup. DN | mAP | |||
| Fixed reliability cost | cosine | 42.45 | 23.60 | 24.14 | |||
| Swap-invariant cost | cosine | 42.33 | 23.95 | 24.11 | |||
| Swap-invariant cost + regression target | ✓ | cosine | 43.65 | 25.54 | 25.50 | ||
| Swap-invariant cost + regression target + supervised DN | ✓ | ✓ | cosine | 50.34 | 32.33 | 31.20 | |
| Swap-invariant cost + regression target + pseudo-label DN | ✓ | ✓ | cosine | 43.25 | 24.90 | 25.07 | |
| All three regression targets | ✓ | ✓ | ✓ | cosine | 49.83 | 31.59 | 30.59 |
| Method or component setting | 10% labeled | 20% labeled | 30% labeled | Fully labeled | ||||||||
| mAP | mAP | mAP | mAP | |||||||||
| DINO | 40.72 | 21.76 | 22.49 | 46.31 | 24.94 | 25.82 | 48.11 | 26.71 | 27.16 | 49.11 | 26.91 | 27.51 |
| Semi-DETR | 43.94 | 25.62 | 25.82 | 48.01 | 28.10 | 28.10 | 50.63 | 29.47 | 29.60 | 51.11 | 31.50 | 30.81 |
| Sparse Semi-DETR | 44.16 | 25.49 | 25.83 | 48.38 | 28.28 | 28.51 | 50.24 | 29.40 | 29.53 | 51.71 | 32.26 | 31.22 |
| Oriented Semi-DETR | 45.92 | 27.77 | 27.61 | 49.89 | 31.67 | 30.69 | 52.11 | 33.51 | 32.19 | 53.16 | 35.07 | 33.23 |
| Early pseudo-label DN ramp | 45.56 | 27.88 | 27.52 | 49.19 | 32.15 | 30.69 | 51.84 | 33.57 | 31.96 | 52.79 | 34.86 | 33.15 |
| Class | 10% labeled | 20% labeled | Fully labeled | ||||||
| Early ramp | + Teacher score | + Encoding margin | Early ramp | + Teacher score | + Encoding margin | Early ramp | + Teacher score | + Encoding margin | |
| plane | 77.01 | 77.18 | 76.92 | 77.19 | 78.65 | 78.88 | 79.41 | 79.16 | 78.98 |
| baseball-diamond | 54.56 | 54.61 | 53.78 | 53.55 | 52.26 | 53.85 | 49.62 | 50.53 | 52.92 |
| bridge | 45.37 | 45.04 | 44.36 | 48.67 | 48.77 | 48.62 | 48.77 | 48.63 | 49.01 |
| ground-track-field | 61.92 | 62.42 | 62.74 | 65.82 | 64.66 | 65.43 | 67.17 | 67.90 | 67.28 |
| small-vehicle | 36.24 | 35.19 | 36.35 | 39.30 | 39.67 | 38.00 | 41.41 | 40.83 | 40.88 |
| Step | Pairs | Sigma | Assign. flip | Filter flip | Keep | Swap only | Fixed only | ||
| fixed | swap | fixed | swap | ||||||
| 20k | 4671 | 0.09% | 16.89% | 18.88% | 7.02% | 35.54% | 36.74% | 4.11% | 2.91% |
| 180k | 3826 | 0.05% | 3.89% | 4.89% | 2.88% | 36.23% | 36.02% | 1.33% | 1.54% |
| Step | Aspect ratio | Sigma | Assign. flip | Filter flip | Keep fixed / swap | |
| 20k | 510 | 43.92% | 45.49% | 18.63% | 28.63% / 44.90% | |
| 1283 | 28.22% | 31.33% | 8.26% | 27.36% / 28.60% | ||
| 2878 | 7.05% | 8.62% | 4.41% | 40.41% / 38.92% | ||
| 180k | 335 | 22.09% | 25.97% | 11.64% | 41.49% / 41.19% | |
| 1109 | 4.24% | 5.50% | 3.16% | 30.21% / 29.58% | ||
| 2382 | 1.18% | 1.64% | 1.51% | 38.29% / 38.29% |
| Step | Class | Sigma | Assign. flip | Filter flip | Keep fixed / swap | |
| 20k | ship | 32 | 34.38% | 37.50% | 9.38% | 37.50% / 46.88% |
| small vehicle | 200 | 16.00% | 16.50% | 5.00% | 55.00% / 56.00% | |
| large vehicle | 680 | 7.06% | 8.97% | 1.62% | 44.41% / 43.38% | |
| swimming pool | 184 | 57.07% | 60.33% | 33.70% | 21.20% / 51.63% | |
| storage tank | 2173 | 13.99% | 16.43% | 6.26% | 36.45% / 34.70% | |
| 180k | ship | 31 | 3.23% | 3.23% | 3.23% | 48.39% / 51.61% |
| Backbone | ResNet-50 [ 4 ] , ImageNet initialized |
| Detector | DINO [ 21 ] , 900 queries, 100 DN queries |
| Optimizer | AdamW, lr , weight decay |
| Backbone lr | the base rate |
| Schedule | 180k iterations, lr at 120k and 160k |
| Batch | 1 labeled and 1 unlabeled image per GPU |
| Train scale | long side 1333, short side sampled in |
Object Detection
Object detection locates and classifies objects in images or video, balancing localization accuracy, robustness, and inference speed.
See all 673 papers