Structure-Aware Multimodal LLM Framework for Trustworthy Near-Field Beam Prediction
Authors: Mengyuan Li, Qianfan Lu, Jiachen Tian, Hongjun Hu, Yu Han, Xiao Li, Chao-kai Wen, Shi Jin
Organizations: School of Information Science and Engineering, Southeast University, Nanjing 210096, China · Institute of Communications Engineering, National Sun Yat-sen University, Kaohsiung 804, Taiwan
In near-field extremely large-scale multiple-input multiple-output (XL-MIMO) systems, spherical wavefront propagation expands the traditional beam codebook into the joint angular-distance domain, rendering conventional beam training prohibitively inefficient, especially in complex 3-dimensional (3D) low-altitude environments. Furthermore, since near-field beam variations are deeply coupled not only with user positions but also with the physical surroundings, precise beam alignment demands profound environmental understanding capabilities. To address this, we propose a large language model (LLM)-driven multimodal framework that fuses historical GPS data, RGB image, LiDAR data, and strategically designed task-specific textual prompts. By utilizing the powerful emergent reasoning and generalization capabilities of the LLM, our approach learns complex spatial dynamics to achieve superior environmental comprehension...
Figures & tables
Fig. 1: Illustration of the XL-MIMO system model in LAE scenarios: The BS is equipped with a UPA, an RGB camera, and a LiDAR, while the UAV is equipped with a GPS which feeds back locations to the BS.
Fig. 2: Overall workflow of the proposed structure-aware LLM-driven multimodal beam prediction framework.
Fig. 3: Architecture of the designed multimodal feature fusion module.
Fig. 4: Architecture of the designed textual prompt encoder and examples of designed textual prompts.
Fig. 5: Architecture of the designed beam prediction head.
Fig. 6: Beam prediction accuracy comparison for different deep learning-based algorithms across overall, LoS, and NLoS scenarios. (a) Top-1 accuracy. (b) Top-5 accuracy.
Fig. 7: Performance comparison of Top-1 beam prediction accuracy against near-field baselines with consistent beam training overhead.
Fig. 8: Achievable rate comparison against baselines and the GT upper bound with consistent beam training overhead.
Configuration
Scenario
Pos MAE [m] ↓
Top-1 Acc [%] ↑
Top-5 Acc [%] ↑
Norm. Gain ↑
High Conf. [%] † ↑
Part I: Ablation of Input Modalities
Full Modalities w/ Adaptive Refinement
Overall
0.8959
82.66 (+39.58)
95.82
0.9462 (+0.2172)
29.4
LoS
0.9032
82.82 (+38.94)
95.90
0.9490 (+0.2052)
30.0
NLoS
0.6675
77.75 (+59.91)
93.43
0.8558 (+0.5923)
13.2
Full Modalities (w/o Adaptive Refinement)
Overall
0.8959
43.08
95.82
0.7290
29.4
LoS
0.9032
43.88
95.90
0.7438
30.0
TABLE I: Performance Comparison and Ablation Study of the Proposed LLM-Driven Multi-Modal Framework.
Fig. 9: Ablation study on the proposed auxiliary trajectory prediction head: Trajectory MAE versus prediction time step τ under various input modality combinations.
Fig. 10: Comparison of the predicted beam index distributions of the test scenes for the proposed framework with and without adaptive refinement, alongside the GT distribution.
Department of Electrical and Computer Engineering, Iowa State University (ISU), Ames, IA, 50014, USA · Gladiolus Technological Institute, USA · School of Electronics and Computer Science, University of Southampton, Southampton SO17 1BJ, U.K. +2
School of Electronic Science and Engineering, South China Normal University, Foshan, China · School of Intelligent Engineering, Shaoguan university, Shaoguan, China · School of Automation, Guangdong University of Technology, Guangzhou, China