Environment-Aware Channel Prediction for Vehicular Communications: A Multimodal Visual Feature Fusion Framework

Xuejian Zhang; Ruisi He; Minseok Kim; Inocent Calist; Mi Yang; Ziyi Qi

cs.CVApr 2, 2026

Environment-Aware Channel Prediction for Vehicular Communications: A Multimodal Visual Feature Fusion Framework

Authors: Xuejian Zhang, Ruisi He, Minseok Kim, Inocent Calist, Mi Yang, Ziyi Qi

Organizations: State Key Laboratory of Advanced Rail Autonomous Operation, the School of Electronics and Information Engineering, and the Frontiers Science Center for Smart High-speed Railway System, Beijing Jiaotong University, Beijing 100044, China · Graduate School of Science and Technology, Niigata University, Niigata, Japan

Abstract

The deep integration of communication with intelligence and sensing, as a defining vision of 6G, renders environment-aware channel prediction a key enabling technology. As a representative 6G application, vehicular communications require accurate and forward-looking channel prediction under stringent reliability, latency, and adaptability demands. Traditional empirical and deterministic models remain limited in balancing accuracy, generalization, and deployability, while the growing availability of onboard and roadside sensing devices offers a promising source of environmental priors. This paper proposes an environment-aware channel prediction framework based on multimodal visual feature fusion. Using GPS data and vehicle-side panoramic RGB images, together with semantic segmentation and depth estimation, the framework extracts semantic, depth, and position features through a three-branch architecture and performs adaptive multimodal fusion via a squeeze-excitation attention gating module. For 360-dimensional angular power spectrum (APS) prediction, a dedicated regression head and a composite multi-constraint loss are further designed. As a result, joint prediction of path loss (PL), delay spread (DS), azimuth spread of arrival (ASA), azimuth spread of departure (ASD), and APS is achieved. Experiments on a synchronized urban V2I measurement dataset yield the best root mean square error (RMSE) of 3.26 dB for PL, RMSEs of 37.66 ns, 5.05 degrees, and 5.08 degrees for DS, ASA, and ASD, respectively, and mean/median APS cosine similarities of 0.9342/0.9571, demonstrating strong accuracy, generalization, and practical potential for intelligent channel prediction in 6G vehicular communications.

Figures & tables

Reference	Scenario	Freqency	Data Modality	Data Source	Sensor Placement *	Target Metric
[ 11 , 12 ]	V2I	28 GHz	RGB Image	Simulation	Tx	Beam & blockage prediction
[ 13 , 14 ]	V2I	60 GHz	RGB Image	Measurement	Tx	Beam prediction
[ 16 ]	Industry	28 GHz	RGB Image, point cloud	Simulation	Factory Ceiling	Beam prediction
[ 17 ]	Indoor	60 GHz	Depth image	Simulation	Tx & Rx	Received Power
[ 18 ]	V2I	60 GHz	RGB image	Measurement	Tx	Received Power
[ 19 ]	V2I	–	RGB image, point cloud, GPS data	Measurement	Tx	PL

TABLE I: Related Work on Environment-Aware Channel Prediction.

Fig. 1: The proposed channel prediction model architecture based on visual sensing and multimodal feature fusion.

Fig. 2: Regression head architecture specifically designed for APS prediction.

Panel of Figure 5 — Fig. 4: Measurement trajectories along three different Areas in urban scenarios. (a) Area 1 ; (b) Area 2 ; (c) Area 3 ; (d) Area 4 .

Parameters	Description
Carrier Frequency	4.85 GHz
Bandwidth	100 MHz
No. Subcarriers	510
Subcarrier Spacing	195 KHz
Sampling Rate	800 MSa/s
Delay Resolution	10 ns

TABLE II: Key Parameters of Measurement Campaign

Fig. 5: Block diagram of the dataset generation pipeline.

Fig. 6: Four representative types of panoramic images: (a) raw panoramic image captured by the rooftop-mounted camera, (b) post-processed image after region cropping, (c) semantic segmented image, (d) depth-estimated image.

Hyperparameter	APS Prediction Model	PL/DS/ASA/ASD Prediction Model
Image Size	224	224
Batch Size	8	16
Max Epochs	100	100
Weight Decay	$10^{-4}$	$10^{-6}$
Gradient Clipping	1.5	1.0
Early Stopping	20 Epochs	20 Epochs

TABLE III: Summary of Training Hyperparameter

Exps.	Semantic Branch Backbone	Input Modal (1)	Evaluation Metrics (2)
			RMSE				MAE				CosSim
			PL [dB]	DS [ns]	ASA [ ∘ ]	ASD [ ∘ ]	PL [dB]	DS [ns]	ASA [ ∘ ]	ASD [ ∘ ]	APS
Exp. 1	ResNet-34	M1 & M2 & M3	3.26	37.66	5.05	5.08	2.08	20.36	2.99	3.23	–
		M1 & M2	5.31	40.30	5.26	5.36	3.12	25.33	3.36	3.68	–
		M1 & M3	4.11	45.09	6.71	5.71	2.89	28.03	5.11	3.58	–
		M1	6.11	40.24	5.48	5.77	4.36	24.10	3.64	3.52	–

TABLE IV: Summary of Results from Different Verification Experiments.

Panel of Figure 11 — Fig. 7: Prediction results of the proposed model for multiple large-scale parameters using semantic segmentation images, depth images, and GPS data inputs, taking (a) DS and (b) ASD as examples.

Panel of Figure 12 — Fig. 8: Results of the impact of different modal inputs on prediction accuracy: (a) PL; (b) DS; (c) ASA; (d) ASD.

Fig. 9: Comparison of the raw image and images with dynamic scatterers removed. (a) raw image; (b) raw image with dynamic scatterers removed; (c) semantic segmentation image with dynamic scatterers removed; (d) depth image with dynamic scatterers removed.

Panel of Figure 14 — Fig. 10: Results of the impact of eliminating dynamic scatterers on prediction accuracy, taking (a) PL and (b) ASA as examples.

Fig. 11: Comparison of training error versus epoch with and without dynamic scatterer removal.

Fig. 12: Results of the impact of different backbone networks on prediction accuracy: taking PL as the example.

Model	Total Params (M)	FLOPs (G)	Avg Inference Time (ms)	Inference FPS
ResNet-34	20.82	7.37	2.82	709.82
VGG-16	120.94	30.93	3.25	616.3
AlexNet	47.26	1.42	0.9	2222.25
U-Net	18.63	26.18	2.71	739.34
MobileNetV2	2.83	664.78	3.09	647.25

TABLE V: Complexity Comparison of Different Semantic Backbones

Panel of Figure 18 — Fig. 13: Examples of APS prediction results.

Fig. 14: Probability density distribution of cosine similarity of all samples in the test set.

Channel State Prediction

See all 14 papers

Low-Altitude Channel Multipath Prediction via Panoramic Perception and Vision-Language Model
Jul 24, 2026Zihang Zeng, Shu Sun, Meixia Tao +3Wireless Communications Radio Propagation Modeling
School of Information Science and Electronic Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
Spatiotemporal Semantic V2X Framework for Cooperative Collision Prediction
Jan 23, 2026Murat Arda Onsu, Poonam Lohan, Burak Kantarci +3Intelligent Transportation Systems Semantic Communication
School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, ON, Canada. · Nokia Bell Labs, 600 March Road, Kanata, ON K2K 2E6, Canada.
DRIFT: Joint Channel Estimation and Prediction Towards Pilotless 6G Non-Terrestrial Networks
May 29, 2026Bruno De Filippo, Carla Amatetti, Alessandro Vanelli-Coralli6G Networks Channel State Prediction
Department of Electrical, Electronic, and Information Engineering (DEI), Univ. of Bologna, Bologna, Italy

Jul 24, 2026J/K move · Enter open · S save

Low-Altitude Channel Multipath Prediction via Panoramic Perception and Vision-Language Model

Zihang Zeng, Shu Sun, Meixia Tao, Zhiyong Chen+2

School of Information Science and Electronic Engineering, Shanghai Jiao Tong University, Shanghai 200240, China

Wireless Communications Radio Propagation Modeling

Open paper AskPDF arXiv

Unmanned aerial vehicle (UAV) communication is expected to support a wide range of low-altitude applications in 6G mobile networks. However, traditional statistical channel models provide limited accuracy in specific environments, while deterministic methods such as ray tracing usually rely on accurate three-dimensional environment models and involve high computational complexity. Existing multimodal channel prediction approaches mainly focus on large-scale metrics such as path loss, and remain insufficient for modeling small-scale parameters. To address these limitations, this paper proposes PanoLAMP, a Panoramic perception and vision-language model-based Low-Altitude Multipath Prediction framework. It adopts a pretrained vision-language model as the backbone and captures the propagation environment features through panoramic RGB-D observations collected at both the transmitter and receiver to predict the delay, power, azimuth angle, and zenith angle offset relative to the line-of-sight path. Experiments are conducted on a synthetic dataset containing 18,949 UAV-vehicle links across seven UAV altitudes. Experimental results show that the proposed method consistently outperforms representative baselines in both multipath parameters and statistical metrics, and demonstrates stronger generalization across different flight heights.

Abstract

Channel State Prediction

Explore similar work

Low-Altitude Channel Multipath Prediction via Panoramic Perception and Vision-Language Model

Spatiotemporal Semantic V2X Framework for Cooperative Collision Prediction

DRIFT: Joint Channel Estimation and Prediction Towards Pilotless 6G Non-Terrestrial Networks