cs.CVAug 16, 2026

CM-MAE: A Physics-Guided Cross-Modal Self-Supervised Learning Framework for Vision-Wireless Applications

Authors: Yubo Zhang, Yiyao Liu

Abstract

Synchronized camera and wireless measurements observe the same scene through different physical channels. The central difficulty is that a representation learned in one deployment can fail when viewpoint, traffic, illumination, and propagation geometry change. This paper presents CM-MAE, a self-supervised vision--wireless pretraining framework for cross-scenario representation transfer. The evaluated real-data model uses only RGB frames and the measured 64-beam received-power vector available in DeepSense 6G; it does not use ray-traced paths, calibrated depth, or beam-index labels during pretraining. Its central pretraining term is a soft contrastive alignment loss. Instead of making the synchronized image--wireless pair the only positive pair, this loss builds a target distribution from similarities between measured beam-power profiles, so nonidentical samples with similar directional responses are not forced apart as false negatives. A masked joint decoder provides the complementary local objective by reconstructing hidden visual patches and wireless angular clusters under modality dropout. After pretraining, a differential-rate fine-tuning rule lets a new fusion head adapt quickly while the encoders move slowly. Under a sequence-disjoint DeepSense 6G protocol, adding the soft alignment loss improves a matched linear-probe transfer average from 24.88% to 29.49%. Mild fusion fine-tuning reaches 77.38% Top-1 accuracy on unseen Scenarios 6--8, and optional transductive normalization adaptation reaches 78.69%. Since the fusion setting uses the contemporaneous 64-beam power vector at inference, these results should be read as representation-transfer diagnostics, not as proactive beam-prediction or reduced-sweeping claims.

Figures & tables

Explore similar work

Oct 13, 2026

Link Both Views: Cross-Image Routing Attention for Dual-Endpoint V2I Beam-Pair Power Prediction

For sensing-aided communications, single-endpoint sensing offers only a partial view of propagation, motivating link-centric sensing across both endpoints. We propose cross-image routing attention (CIRA) for sparse association and mutual enhancement of propagation-relevant features from dual-endpoint RGB images. Building on CIRA, CiFormer predicts millimeter-wave (mmWave) beam-pair power matrices for vehicle-to-infrastructure (V2I) communications. Experiments show that CiFormer reduces mean absolute error and improves top-1 beam-pair selection accuracy by 3.15 percentage points over the strongest baseline. Visualizations and ablation studies support dual-endpoint sensing and selective cross-view interaction in capturing propagation-relevant dependencies, suggesting broader applications in wireless tasks beyond beam prediction.
Jul 1, 2026

Meta-Transfer Learning for mmWave Beam Alignment

Millimeter-wave (mmWave) beam alignment is critical for next-generation wireless systems, but existing approaches either update all parameters during adaptation or restrict updates to a subset of layers without adapting intermediate feature representations. We propose MTL-BA, a meta-transfer learning framework for multiple-input single-output (MISO) beam alignment that freezes a pre-trained convolutional backbone and meta-learns lightweight Scale-and-Shift (SS) adapters and a classifier head. Under an outdoor-to-indoor shift in DeepMIMO, MTL-BA achieves the highest Top-1 accuracy among the evaluated methods at 35 dB while updating 17×17\times fewer parameters than full fine-tuning and meta-training 6×6\times faster than MAML.
Sep 4, 2026

Wireless Foundation Models: State-of-the-Art and Open Challenges

Wireless foundation models (WFMs) have emerged as a promising approach for learning reusable representations from large-scale wireless data and adapting them to downstream tasks. However, the rapidly growing literature remains fragmented across modalities, pretraining objectives, architectures, adaptation strategies, and evaluation protocols, making it difficult to assess progress toward broadly transferable models. This survey provides a systematic analysis of WFMs for physical-layer applications. We first introduce the main WFM design components, including pretraining, backbone architectures, and downstream adaptation. We then organize the literature into five physical-layer task families: signal recognition and demodulation, channel representation learning, RF sensing and localization, beam management, and spectrum sensing and monitoring, while separately examining multi-task PHY models. Across these categories, we analyze how existing models are pretrained, adapted, and evaluated, with particular attention to downstream task diversity and the distinction between in-distribution, partial-shift, and out-of-distribution transfer. Our analysis shows that current WFMs provide increasing evidence of reusable wireless representations, but this evidence varies considerably across task families and evaluation settings. Differences in datasets, modalities, architectures, pretraining objectives, adaptation protocols, and distribution shifts make it difficult to determine which design choices drive transfer and generalization. We conclude by identifying open directions for improving data availability, evaluation rigor, generalization, efficient adaptation, and real-world deployment, providing a unified framework for understanding the current WFM landscape and the requirements for developing more reusable foundation models for future physical-layer wireless systems.