Multimodal Road Closure Mapping via Cross-Modal Graph Attention
Organizations: Amazon, US
Abstract
Accurate and timely identification of road closures is critical for large-scale navigation and logistics systems, yet remains challenging due to the need to jointly reason over visual evidence, spatial context, and road network topology. Existing approaches either rely on manual map updates or focus on detecting closure-related objects without resolving which road segment is affected. In this work, we introduce a unified multimodal and multitask framework that jointly performs road closure classification (no, partial, complete) and road segment association within a single learning architecture. Our model integrates graph-structured map data, satellite imagery augmented with projected street-level detections, and street-view images using modality-specific encoders and cross-modal transformer attention, enabling explicit reasoning over visual cues and network topology. Evaluated on nearly 20K real-world scenarios collected from a large-scale commercial vehicle fleet, our approach achieves 72.7% closure classification accuracy and 60% road segment association accuracy, outperforming frontier vision-language models by over 28 percentage points on closure classification while being over 4,000x smaller and 100x faster at inference. Extensive ablations demonstrate the importance of cross-modal attention, edge-aware pooling, and graph message passing for accurate geospatial reasoning. This work establishes the first benchmark for joint road closure classification and segment-level attribution, and shows that task-specific architectural inductive biases substantially outperform general-purpose multimodal models for structured geospatial understanding.
Figures & tables
| Model (params) | Closure/Assoc. Acc. (%) | Latency (s) |
|---|---|---|
| Nearest-Edge Heuristic (0) | — / 45.4 | — |
| VLM-A ( 100B) | 34.7 / — | 7.0 |
| VLM-B ( 100B) | 44.3 / — | 5.4 |
| Ours (Separate) (12M each) | 71.6 / 58.3 | 0.06 |
| Ours (Joint) (23M) | 72.7 / 59.9 | 0.05 |
| Sat. | Sat.+Proj. | Graph | Street | P / R / F1 (%) | # Params |
|---|---|---|---|---|---|
| ✗ | ✗ | ✓ | ✗ | 39.4 / 32.9 / 35.9 | 218K |
| ✓ | ✗ | ✗ | ✗ | 22.6 / 26.9 / 24.6 | 12.32M |
| ✓ | ✓ | ✗ | ✗ | 34.0 / 30.6 / 32.2 | 12.32M |
| ✓ | ✓ | ✓ | ✗ | 52.9 / 38.0 / 44.2 | 12.37M |
| ✓ | ✓ | ✓ | ✓ | 58.0 / 37.5 / 45.5 | 23.42M |
| Component | Variant | P / R / F1 (%) |
| Fusion Strategy | Early Fusion | 41.2 / 38.1 / 39.5 |
| Late Fusion | 47.8 / 38.8 / 42.8 | |
| Cross-Attention | 58.0 / 37.5 / 45.5 | |
| Graph Pooling | Global | 43.4 / 37.9 / 40.4 |
| Edge-Aware | 58.0 / 37.5 / 45.5 | |
| Graph Encoder | GAT | 41.0 / 36.9 / 38.8 |
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
| Component | Parameter | Value |
| Global Settings | ||
| Hidden dimension ( ) | 512 | |
| Dropout rate | 0.2 | |
| Graph Encoder (GATv2) | ||
| Number of GAT layers | 3 | |
| Number of attention heads | 8 | |
| Parameter | Value |
|---|---|
| Optimizer | AdamW |
| Learning rate | |
| Weight decay | |
| Batch size | 32 |
| Number of epochs | 60 |
| Learning rate scheduler | ReduceLROnPlateau |
| Parameter | Value/Method |
|---|---|
| Image Normalization | |
| Mean (RGB) | [0.485, 0.456, 0.406] |
| Std (RGB) | [0.229, 0.224, 0.225] |
| Graph Features | |
| Road class normalization | Ordinal (10-102 0-1) |
| Distance normalization | Min-max |
| Resource | Specification |
|---|---|
| GPU | Tesla V100 (16GB) |
| Number of GPUs | 1 |
| Training time per epoch | 4 minutes |
| Total training time | 4 hours |
Multimodal Fusion
Multimodal fusion combines information from different data types to improve prediction, perception, and decision-making.
See all 805 papers