In collaborative perception, an agent's performance can be degraded by heterogeneity arising from differences in model architecture or training data distributions. To address this challenge, we propose HyDRA (Hybrid Domain-Aware Robust Architecture), a unified pipeline that integrates intermediate and late fusion within a domain-aware framework. We introduce a lightweight domain classifier that dynamically identifies heterogeneous agents and assigns them to the late-fusion branch. Furthermore, we propose anchor-guided pose graph optimization to mitigate localization errors inherent in late fusion, leveraging reliable detections from intermediate fusion as fixed spatial anchors. Extensive experiments demonstrate that, despite requiring no additional training, HyDRA achieves performance comparable to state-of-the-art heterogeneity-aware CP methods. Importantly, this performance is maintained as the number of collaborating agents increases, enabling zero-cost scaling without retraining.
Figures & tables
Fig. 1: Heterogeneity in dynamic collaboration. (b) Architecture Heterogeneity occurs when agents employ diverse model architectures, making direct feature fusion structurally infeasible. (c) Latent Domain Heterogeneity is from an independent training setting even when agents adopt identical architectures, resulting in a more subtle threat. Our proposed framework successfully identifies not only explicit structural mismatches but also these hidden latent domain shifts to prevent feature contamination.
Fig. 2: Overview of HyDRA . Domain Classifier computes a domain similarity score Sdomain by comparing the ego-decoded prediction Bpred with the received auxiliary detection result BA . In Hybrid Fusion , homogeneous agents ( Nint ) participate in intermediate fusion to generate Stage 1 detections ( Bstage1 ), while heterogeneous agents ( Nlate ) participate in late fusion using their transmitted detection results. At the late-fusion stage, Anchor-Guided Pose Graph Optimization mitigates pose noise by treating the reliable Stage 1 results as fixed spatial anchors to correct the poses of heterogeneous agents.
Fig. 3: Concept of AG-PGO . Instead of global optimization, we leverage the reliable detections from homogeneous agents ( Nint ) as fixed spatial anchors ( ). The module corrects the pose of heterogeneous agents ( Nlate ) by minimizing the residual errors between their predictions and the anchors, ensuring global consistency.
TABLE I: Performance comparison under architecture heterogeneity : EP+HP+XS+XV . Best and second-best results are highlighted in bold and underline , respectively.
Fig. 4: Performance comparison with baseline methods under varying pose noise and scalability analysis
Collaborative Perception (CP) improves autonomous systems' awareness of their surroundings by sharing sensor data, intermediate features, and detection results. In real-world deployments, however, collaborating vehicles often use heterogeneous sensors, perception models, datasets, and training domains, creating feature-space shifts that degrade downstream fusion and detection. Existing approaches typically retrain fusion and detection components or introduce modality-specific feature interpreters. These methods scale poorly to newly joining agents and often require access to proprietary metadata, raising privacy concerns. We propose HeteroPROMPT, a real-time and privacy-preserving framework for heterogeneous collaborative perception. HeteroPROMPT rapidly aligns each heterogeneous agent's features with an ego-centric unified feature space through modular prompts and lightweight learning-based tuning, while keeping agent encoders and the collaborative fusion and detection stacks frozen. Its visual prompt-based training and inference modulate Bird's Eye View (BEV) features across channels and spatial locations with low computational overhead. For metadata-free deployment, an autoencoder learns a compact unified representation and extracts modality cues from shared features, enabling real-time modality classification and routing to the appropriate HeteroPROMPT modules without exposing proprietary agent information. Experiments on the OPV2V-H and V2XSet datasets show that HeteroPROMPT improves Average Precision over state-of-the-art heterogeneous CP methods while using orders of magnitude fewer trainable parameters. This offers a scalable and practical CP solution. The proposed modality classifier also predicts the joining agent's modality from compact features with greater than 99.99 percent accuracy during deployment. Code will be available at https://github.com/arminmaleki007/HeteroPROMPT.
Armin Maleki, Hayder Radha
Electrical and Computer Engineering, Michigan State University
In autonomous driving, multi-agent collaborative perception enhances sensing capabilities by enabling agents to share perceptual data. A key challenge lies in handling {\em heterogeneous} features from agents equipped with different sensing modalities or model architectures, which complicates data fusion. Existing approaches often require retraining encoders or designing interpreter modules for pairwise feature alignment, but these solutions are not scalable in practice. To address this, we propose {\em GT-Space}, a flexible and scalable collaborative perception framework for heterogeneous agents. GT-Space constructs a common feature space from ground-truth labels, providing a unified reference for feature alignment. With this shared space, agents only need a single adapter module to project their features, eliminating the need for pairwise interactions with other agents. Furthermore, we design a fusion network trained with contrastive losses across diverse modality combinations. Extensive experiments on simulation datasets (OPV2V and V2XSet) and a real-world dataset (RCooper) demonstrate that GT-Space consistently outperforms baselines in detection accuracy while delivering robust performance. Our code will be released at https://github.com/KingScar/GT-Space.
Wentao Wang, Haoran Xu, Guang Tan
Sun Yat-sen University, Shenzhen Campus, Shenzhen, China · Peng Cheng Laboratory (PCL), Shenzhen, China
Collaborative perception (CP) is a promising paradigm for improving situational awareness in autonomous vehicles by overcoming the limitations of single-agent perception. However, most existing approaches assume homogeneous agents, which restricts their applicability in real-world scenarios where vehicles use diverse sensors and perception models. This heterogeneity introduces a feature domain gap that degrades detection performance. Prior works address this issue by retraining entire models/major components, or using feature interpreters for each new agent type, which is computationally expensive, compromises privacy, and may reduce single-agent accuracy. We propose Faster-HEAL, a lightweight and privacy-preserving CP framework that fine-tunes a low-rank visual prompt to align heterogeneous features with a unified feature space while leveraging pyramid fusion for robust feature aggregation. This approach reduces the trainable parameters by 94%, enabling efficient adaptation to new agents without retraining large models. Experiments on the OPV2V-H dataset show that Faster-HEAL improves detection performance by 2% over state-of-the-art methods with significantly lower computational overhead, offering a practical solution for scalable heterogeneous CP.
Armin Maleki, Hayder Radha
Electrical and Computer Engineering, Michigan State University, East Lansing, Michigan