Environment-Aware Channel Prediction for Vehicular Communications: A Multimodal Visual Feature Fusion Framework
Organizations: State Key Laboratory of Advanced Rail Autonomous Operation, the School of Electronics and Information Engineering, and the Frontiers Science Center for Smart High-speed Railway System, Beijing Jiaotong University, Beijing 100044, China · Graduate School of Science and Technology, Niigata University, Niigata, Japan
Abstract
The deep integration of communication with intelligence and sensing, as a defining vision of 6G, renders environment-aware channel prediction a key enabling technology. As a representative 6G application, vehicular communications require accurate and forward-looking channel prediction under stringent reliability, latency, and adaptability demands. Traditional empirical and deterministic models remain limited in balancing accuracy, generalization, and deployability, while the growing availability of onboard and roadside sensing devices offers a promising source of environmental priors. This paper proposes an environment-aware channel prediction framework based on multimodal visual feature fusion. Using GPS data and vehicle-side panoramic RGB images, together with semantic segmentation and depth estimation, the framework extracts semantic, depth, and position features through a three-branch architecture and performs adaptive multimodal fusion via a squeeze-excitation attention gating module. For 360-dimensional angular power spectrum (APS) prediction, a dedicated regression head and a composite multi-constraint loss are further designed. As a result, joint prediction of path loss (PL), delay spread (DS), azimuth spread of arrival (ASA), azimuth spread of departure (ASD), and APS is achieved. Experiments on a synchronized urban V2I measurement dataset yield the best root mean square error (RMSE) of 3.26 dB for PL, RMSEs of 37.66 ns, 5.05 degrees, and 5.08 degrees for DS, ASA, and ASD, respectively, and mean/median APS cosine similarities of 0.9342/0.9571, demonstrating strong accuracy, generalization, and practical potential for intelligent channel prediction in 6G vehicular communications.
Figures & tables
| Reference | Scenario | Freqency | Data Modality | Data Source | Sensor Placement * | Target Metric |
| [ 11 , 12 ] | V2I | 28 GHz | RGB Image | Simulation | Tx | Beam & blockage prediction |
| [ 13 , 14 ] | V2I | 60 GHz | RGB Image | Measurement | Tx | Beam prediction |
| [ 16 ] | Industry | 28 GHz | RGB Image, point cloud | Simulation | Factory Ceiling | Beam prediction |
| [ 17 ] | Indoor | 60 GHz | Depth image | Simulation | Tx & Rx | Received Power |
| [ 18 ] | V2I | 60 GHz | RGB image | Measurement | Tx | Received Power |
| [ 19 ] | V2I | – | RGB image, point cloud, GPS data | Measurement | Tx | PL |
| Parameters | Description |
| Carrier Frequency | 4.85 GHz |
| Bandwidth | 100 MHz |
| No. Subcarriers | 510 |
| Subcarrier Spacing | 195 KHz |
| Sampling Rate | 800 MSa/s |
| Delay Resolution | 10 ns |
| Hyperparameter | APS Prediction Model | PL/DS/ASA/ASD Prediction Model |
| Image Size | 224 | 224 |
| Batch Size | 8 | 16 |
| Max Epochs | 100 | 100 |
| Weight Decay | ||
| Gradient Clipping | 1.5 | 1.0 |
| Early Stopping | 20 Epochs | 20 Epochs |
| Exps. | Semantic Branch Backbone | Input Modal (1) | Evaluation Metrics (2) | ||||||||
| RMSE | MAE | CosSim | |||||||||
| PL [dB] | DS [ns] | ASA [ ∘ ] | ASD [ ∘ ] | PL [dB] | DS [ns] | ASA [ ∘ ] | ASD [ ∘ ] | APS | |||
| Exp. 1 | ResNet-34 | M1 & M2 & M3 | 3.26 | 37.66 | 5.05 | 5.08 | 2.08 | 20.36 | 2.99 | 3.23 | – |
| M1 & M2 | 5.31 | 40.30 | 5.26 | 5.36 | 3.12 | 25.33 | 3.36 | 3.68 | – | ||
| M1 & M3 | 4.11 | 45.09 | 6.71 | 5.71 | 2.89 | 28.03 | 5.11 | 3.58 | – | ||
| M1 | 6.11 | 40.24 | 5.48 | 5.77 | 4.36 | 24.10 | 3.64 | 3.52 | – | ||
| Model | Total Params (M) | FLOPs (G) | Avg Inference Time (ms) | Inference FPS |
| ResNet-34 | 20.82 | 7.37 | 2.82 | 709.82 |
| VGG-16 | 120.94 | 30.93 | 3.25 | 616.3 |
| AlexNet | 47.26 | 1.42 | 0.9 | 2222.25 |
| U-Net | 18.63 | 26.18 | 2.71 | 739.34 |
| MobileNetV2 | 2.83 | 664.78 | 3.09 | 647.25 |