CoMET: Tracking the Shared Pose Error of Roadside Messages for Asynchronous Vehicle-Infrastructure Cooperative 3D Detection
Organizations: Xi’an Jiaotong University
Abstract
A roadside sensor can detect vehicles hidden from the ego vehicle, but a pose error shifts every detection in the same message. We study asynchronous vehicle-infrastructure cooperative 3D detection on V2X-Seq using only information available at decision time, without ground truth, future frames, or per-pair calibration offsets at inference. Objects seen by both sensors reveal the shared translation, but even at zero added delay, 38% of validation messages have no matched pair. We propose CoMET (Common-Mode Error Tracking), which estimates this translation over time. At each message arrival, CoMET matches vehicle and roadside detections, estimates the offset from mutually nearest pairs, and updates a Kalman filter for that sender. When no pair matches, the filter carries its estimate forward. The correction applies to all roadside boxes, including those unseen by the ego vehicle. The estimator has no learned parameters; a separate frozen head ranks the merged detections. Across four delays from 0 to 400 ms on 21 validation sequences, CoMET averages 66.63 bird's-eye-view AP at IoU 0.7, compared with 63.18 for the same late fusion using uncorrected online poses and 62.96 for the best intermediate-fusion baseline. Across five detectors from three architectures, CoMET gains 2.2 to 3.5 AP points over online poses and 0.3 to 0.6 over estimating the offset from the current message alone. We find that the benchmark uses a zero offset as a placeholder in 28% of validation messages and that its offset search reaches the boundary in 45% of the rest. These offsets affect the labels for objects seen only from the roadside. Under two alternative label references, CoMET improves on uncorrected online poses in every evaluated population. Retraining intermediate fusion with CoMET poses and corrected labels improves detection on all three architectures tested, with statistically significant gains on two.
Figures & tables
| Split | Messages | Exactly zero | Zero with co-labelled obj. | Non-zero without co-labelled obj. | Non-zero on a bound |
| fit (24 seq.) | 3762 | 1001 (26.6%) | 0 | 16 | 1379 (49.9%) |
| calibration (8) | 1247 | 319 (25.6%) | 35 | 1 | 721 (77.7%) |
| held-out (14) | 2436 | 530 (21.8%) | 0 | 10 | 810 (42.5%) |
| validation (21) | 3316 | 928 (28.0%) | 0 | 11 | 1066 (44.6%) |
| [email protected] | [email protected] | |||||||||
| Method | 0 | 100 | 200 | 400 | mean | 0 | 100 | 200 | 400 | mean |
| Ego detector only | 62.15 | 62.15 | 62.15 | 62.15 | 62.15 | 70.93 | 70.93 | 70.93 | 70.93 | 70.93 |
| F-Cooper [ 2 ] | 62.31 | 62.20 | 62.20 | 62.01 | 62.18 | 77.33 | 77.10 | 76.85 | 76.48 | 76.94 |
| TraF-Align [ 15 ] | 60.04 | 61.00 | 61.16 | 62.26 | 61.11 | 75.11 | 75.37 | 75.18 | 75.41 | 75.27 |
| V2X-ViT [ 20 ] (3 seeds) | 63.11 | 62.93 | 62.91 | 62.91 | 62.96 | 78.31 | 77.98 | 77.64 | 77.32 | 77.81 |
| Official late fusion [ 27 ] (best of two) | 63.06 | 62.92 | 62.85 | 62.55 | 62.84 | 74.23 | 74.03 | 73.84 | 73.07 | 73.79 |
| validation | held-out | |||||
| Detector | CoMET | online | single | CoMET | online | single |
| TraF-Align (released) | 66.63 | +3.45 | +0.57 | 72.57 | +8.37 | +1.38 |
| TraF-Align (strict retrain) | 58.96 | +2.74 | +0.30 | 61.57 | +6.82 | +1.16 |
| V2X-ViT, seed A | 66.08 | +2.38 | +0.49 | 71.30 | +5.92 | +1.24 |
| V2X-ViT, seed B | 64.97 | +2.22 | +0.42 | 68.95 | +4.12 | +0.83 |
| Where2comm | 65.14 | +3.05 | +0.46 | 71.29 | +6.73 | +1.19 |
| validation | held-out | |||
| Estimator of | [email protected] | [email protected] | [email protected] | [email protected] |
| Current message, mean | 66.06 | 78.17 | 71.19 | 85.93 |
| Current message, median | 66.05 | 78.14 | 71.29 | 85.90 |
| Hold last observation | 65.99 | 78.26 | 71.05 | 85.80 |
| Exponential moving average | 66.17 | 78.50 | 72.49 | 87.03 |
| Sliding-window mean | 66.63 | 78.71 | 72.07 | 86.77 |
| Anchor candidates | Association | calibration | validation | held-out | |
| N0 | confident ego anchors (score ) | Hungarian + mutual NN | 66.39 | 64.15 | 68.94 |
| U2 | confident ego anchors | Sinkhorn | 65.72 | 64.12 | 68.59 |
| U3 | confident ego anchors | Hungarian, prediction-gated | 66.02 | 64.14 | 68.34 |
| U0 | all ego boxes | Sinkhorn, m, centred on estimate | 66.12 | 64.40 | 68.79 |
| U1 | all ego boxes | Sinkhorn, m, uncentred | 66.28 | 63.92 | 68.04 |
| U5 | all ego boxes | Sinkhorn, m | 65.87 | 64.16 | 69.09 |
| Anchor score threshold | all | 0.1 | 0.2 | 0.3 | 0.4 | 0.5 | orig. |
| [email protected] | 66.63 | 66.63 | 66.63 | 66.50 | 66.25 | 65.99 | 65.98 |
| Association gate (m) | 1.5 | 2.0 | 2.5 | 3.0 | |||
| [email protected] | 66.21 | 66.65 | 66.63 | 66.66 |
| Module | per frame | + temporal median | + gated KF |
| FreeAlign | 63.49 | 63.54 | 63.88 |
| RoCo | 64.36 | 64.30 | 64.20 |
| CBM | 64.06 | 64.00 | 63.88 |
| Relative pose | 0 | 100 | 200 | 400 | mean | [email protected] |
| CoAlign pose graph [ 13 ] | 56.99 | 56.64 | 56.47 | 56.26 | 56.59 | 72.30 |
| Anchor-gated KF | 57.95 | 57.74 | 57.52 | 57.21 | 57.60 | 73.67 |
| CoMET | 58.06 | 57.82 | 57.59 | 57.33 | 57.70 | 73.74 |
| CoMET online pose | CoMET single message | |||||
| Detector | all | co-lab. | zero | all | co-lab. | zero |
| TraF-Align (released) | +3.45 | +4.53 | 3.42 | +0.57 | +0.98 | 2.02 |
| TraF-Align (strict) | +2.74 | +3.47 | 2.01 | +0.30 | +0.53 | 1.10 |
| V2X-ViT A | +2.38 | +3.16 | 2.61 | +0.49 | +0.62 | 0.28 |
| V2X-ViT B | +2.22 | +2.93 | 2.32 | +0.42 | +0.55 | 0.35 |
| Where2comm | +3.05 | +3.97 | 2.73 | +0.46 | +0.57 | 0.15 |
| interpolated | label-median | |||
| Split | Difference | TraF-Align (released) | TraF-Align (released) | TraF-Align (strict) |
| validation | CoMET online pose | +4.00 [2.41, 5.69] | +3.93 [2.26, 5.78] | +2.84 [1.66, 4.08] |
| validation | CoMET current message | +0.94 [0.58, 1.33] | +0.90 [0.50, 1.35] | +0.35 [0.09, 0.60] |
| held-out | CoMET online pose | +8.73 [2.51, 12.52] | +9.30 [2.65, 13.43] | +7.32 [1.95, 10.57] |
| held-out | CoMET current message | +1.58 [0.18, 2.58] | +1.42 [0.05, 2.42] | +1.10 [0.16, 1.78] |
| Training and test pose / training labels | [email protected] | [email protected] |
| Online pose, original labels | 65.68 | 80.26 |
| Anchor-gated pose, original labels | 66.18 | 81.19 |
| Anchor-gated pose, cleaned labels | 66.44 | 81.36 |
| CoMET pose, cleaned labels (tier 2) | 66.49 | 81.44 |
| CoMET without retraining (tier 1) | 66.63 | 78.87 |
| Architecture | online | anchor-gated | tier 2 | [email protected] | [email protected] |
| Where2comm | 63.05 / 76.96 | 63.03 / 77.07 | 63.29 / 77.28 | +0.24 [ 0.16, 0.54] | +0.32 [ 0.29, 0.84] |
| CoAlign | 56.47 / 72.13 | 57.85 / 73.78 | 58.10 / 73.88 | +1.63 [1.20, 2.10] | +1.75 [1.03, 2.62] |
| ms | ||||||
| Pose error | stale | CoMET | oracle | stale | CoMET | oracle |
| structured, fitted to V2X-Seq | 73.22 | 82.14 | 85.07 | 66.90 | 70.73 | 74.62 |
| i.i.d. per frame (0.2 m, 0.2 ∘ ) | 80.50 | 81.10 | 85.07 | 67.43 | 69.19 | 74.62 |
| ego pose time error 50 ms | 77.70 | 80.41 | 85.07 | 67.91 | 72.58 | 74.62 |
| ego pose time error 100 ms | 70.82 | 76.36 | 85.07 | 67.64 | 71.35 | 74.62 |
| ego pose time error 150 ms | 68.58 | 74.42 | 85.07 | 67.56 | 70.49 | 74.62 |
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
| Split | 0 ms | 100 ms | 200 ms | 400 ms |
| fit | 3.31 / 29.8% | 3.28 / 30.0% | 3.20 / 31.1% | 2.98 / 33.1% |
| calibration | 3.03 / 33.5% | 2.97 / 34.6% | 3.00 / 35.0% | 2.89 / 34.2% |
| held-out | 3.34 / 31.7% | 3.35 / 30.8% | 3.30 / 31.2% | 3.11 / 32.3% |
| validation | 2.96 / 38.2% | 2.92 / 38.7% | 2.90 / 37.6% | 2.73 / 38.5% |
| validation | held-out | |||
| Re-registered | CoMET online | CoMET single | CoMET online | CoMET single |
| gaps 1 s | +3.66 [1.93, 5.48] | +0.60 [0.06, 1.13] | +8.43 [1.80, 12.41] | +1.43 [ 0.15, 2.53] |
| gaps 3 s | +3.66 [1.93, 5.48] | +0.60 [0.06, 1.13] | +8.45 [1.89, 12.45] | +1.45 [ 0.11, 2.55] |
| all gaps | +4.00 [2.41, 5.69] | +0.94 [0.58, 1.33] | +8.73 [2.51, 12.52] | +1.58 [0.18, 2.58] |
| CoMET online | CoMET single | ||||||
| Detector | Split | all | co-lab. | zero | all | co-lab. | zero |
| TraF-Align (released) | val. | +3.93 | +4.45 | +0.83 | +0.90 | +0.94 | +0.77 |
| TraF-Align (released) | held-out | +9.30 | +10.21 | +0.58 | +1.42 | +1.56 | +0.22 |
| TraF-Align (strict) | val. | +2.84 | +3.24 | +0.38 | +0.35 | +0.40 | +0.13 |
| TraF-Align (strict) | held-out | +7.32 | +8.10 | +0.58 | +1.10 | +1.22 | +0.22 |
| Training and test pose / training labels | [email protected] | [email protected] |
| Online pose, original labels | 68.30 | 87.68 |
| Anchor-gated pose, original labels | 72.15 | 90.09 |
| Anchor-gated pose, cleaned labels | 72.73 | 89.94 |
| CoMET pose, cleaned labels (tier 2) | 71.45 | 89.59 |
| CoMET without retraining (tier 1) | 72.57 | 86.99 |
| val. [email protected] | val. [email protected] | held-out [email protected] | held-out [email protected] | |
| A online-pose fine-tune | +0.81 [0.18, 1.62] | +1.17 [0.58, 1.87] | +3.15 [0.23, 4.81] | +1.91 [ 0.67, 3.05] |
| B online-pose fine-tune | +1.08 [0.37, 1.94] | +1.40 [0.81, 2.10] | +3.74 [0.55, 5.38] | +2.13 [ 0.02, 3.12] |
| Reference | Difference | val. [email protected] | val. [email protected] | held-out [email protected] | held-out [email protected] |
| interpolated | tier 2 online-pose fine-tune | +1.19 [0.54, 2.00] | +1.79 [1.11, 2.57] | +3.43 [0.90, 4.92] | +2.40 [0.46, 3.26] |
| label-median | tier 2 online-pose fine-tune | +1.27 [0.56, 2.07] | +2.29 [1.25, 3.72] | +4.00 [1.26, 5.66] | +3.25 [1.52, 4.15] |
Cooperative Perception
Cooperative perception lets connected vehicles, robots, and aerial agents combine sensor observations to understand scenes beyond a single viewpoint.
See all 119 papers