cs.CVSep 19, 2026
SaveGeneral Collaborative Intelligence: Architecting Cognition for Resilient Multi-Agent Ecosystems
Organizations: School of Astronautics, Northwestern Polytechnical University, Xi’an, China. · School of Computer Science, Wuhan University, Wuhan, China. · School of Electronics and Control Engineering, Chang’an University, Xi’an, China. · School of Automation, Northwestern Polytechnical University, Xi’an, China. · College of Computer Science, Nankai University, Tianjin, China.
Abstract
Multi-agent unmanned systems are moving from isolated, ego-centric sensing toward collaborative intelligence, in which distributed agents exchange compact features to overcome a local observation trap that no single agent can escape: occlusions, finite sensor range, and environmental degradation. The field has matured across architectural, communication, embodied, resilience, and trust dimensions, yet existing surveys examine these dimensions in isolation and rarely expose their dependencies. This review offers a unified synthesis through two complementary lenses. The first is a five-dimensional taxonomy spanning collaboration stage, communication paradigm, fusion architecture, learning strategy, and application domain. The second is three cognitive synergy conditions, Semantic Disambiguation, Pragmatic Information Exchange, and Proactive Informational Foraging, that turn cognitive synergy into operational criteria. Across these lenses we survey collaboration architectures and topologies, neural-communication co-design that treats the channel as a differentiable pipeline component, embodied action-perception loops via multi-agent reinforcement learning, and resilience mechanisms for synchronization, uncertainty quantification, and label-efficient learning. We then map these advances onto four operational domains, V2X, unmanned aerial, industrial logistics, and smart cities, and onto the safety-privacy-utility triad. To counter benchmark saturation and evaluation fragmentation, we propose GCI-Bench, a five-pillar scoring protocol with a maturity model that makes the trade-offs of collaborative methods comparable across studies. A critical reflection on reproducibility, the sim-to-real gulf, and conditions under which collaboration degrades performance identifies open challenges and charts directions toward general collaborative intelligence under real-world uncertainty.
Figures & tables
Figure 1: Vision of general collaborative intelligence for multi-agent unmanned systems. The transition from isolated autonomy to GCI: heterogeneous agents exchange compact feature tokens through communication-aware fusion, evolving from deterministic data fusion toward cognitive synergy in a distributed ecosystem supporting open-world adaptation, pragmatic communication, embodied information foraging, and trustworthy synergy.
| Application Domain | Technical Core | Tasks & Robustness | System & Reflection | |||||||
| Ref. | Year | V2X | UxS/ISC | Co-Des. | Formal. | Tasks | Chall. | Embod. | Trust | Refl. |
| Han et al. [ 14 ] | 2023 | – | – | – | – | – | – | |||
| Yazgan et al. [ 11 ] | 2024 | – | – | – | ||||||
| Bai et al. [ 15 ] | 2024 | – | – | – | – | – | – | |||
| Song et al. [ 10 ] | 2025 | – | – | – | – | – | ||||
| Huang et al. [ 3 ] | 2025 | – | – | – | – | |||||
Table 1: Systematic comparison of this review with representative prior surveys. A filled circle ( ) denotes comprehensive coverage; an open circle ( ) denotes partial coverage; a dash (–) indicates the dimension is not addressed.
| Condition | Observable Criterion | Typical Technical Realization | Operational Failure Test (metric; benchmark) |
|---|---|---|---|
| C1 Semantic Disambiguation | Resolve entity identity, spatial location, and trajectory intent beyond any single agent’s line-of-sight | Cross-attention Transformers [ 23 , 24 ] ; generative map priors (CoGMP) [ 25 ] | On a dynamic-occlusion split of OPV2V [ 26 ] : identity-switch or class-misattribution rate above a set threshold (e.g., 5%) as targets cross multi-agent occlusions |
| C2 Pragmatic Exchange | Bandwidth scales inversely with spatial confidence; high-entropy, high-surprise regions transmitted first | Extended information bottleneck (InfoCom [ 27 ] ); spatial confidence masking (Where2comm [ 21 ] ) | On OPV2V [ 26 ] or V2X-Sim [ 28 ] : communication volume scaling near-linearly with agent density (log–log slope ), or a non-negligible share of redundant background bytes |
| C3 Proactive Foraging | Trajectory adjustments or sensor re-aiming triggered by localized state-uncertainty peaks | Active perception loops; planning-integrated perception (Plan2comm [ 29 ] , CooperDrive [ 30 ] ) | On an active-perception testbed (e.g., CSAOT [ 31 ] ): no measurable trajectory or sensor reconfiguration, and hence no entropy reduction, in regions of high epistemic uncertainty |
Table 2: Operational criteria for cognitive synergy. The three conditions translate the conceptual claim of GCI into observable requirements that can be evaluated experimentally.
Figure 2: A multi-dimensional taxonomy of collaborative perception. The taxonomy organizes the field along five orthogonal dimensions. Dimension 1 (Collaboration Stage) determines the abstraction level at which information is exchanged, directly governing the rate–fidelity trade-off formalized in Eq. 1 . Dimension 2 (Communication Paradigm) spans the spectrum from periodic broadcasting to neural-communication co-design (Section 4 ). Dimension 3 (Fusion Architecture) specifies the topological structure of information aggregation. Dimension 4 (Learning Strategy) determines how the collaborative model acquires its parameters. Dimension 5 (Application Domain) captures the operational context that drives domain-specific design choices (Section 7 ).
Table 3: Taxonomy of collaboration architectures and cognitive synergy conditions. Methods are mapped onto the proposed five-dimensional framework.
Figure 3: Illustration of three primary collaboration strategies in multi-agent perception systems. The collaboration stage dictates the abstraction level at which information is exchanged. Early Collaboration (solid line) shares high-bandwidth raw data before the feature backbone. Intermediate Collaboration (dashed line) shares compressed features after the backbone, offering a balance between performance and communication cost. Late Collaboration (dotted line) shares low-bandwidth local outputs (e.g., object detections) after the perception head. Lower panels illustrate the fusion process within each strategy, where data from multiple agents (differentiated by color) is aggregated into a unified representation before being re-injected into the ego-agent’s pipeline.
Figure 4: Canonical collaborative perception architecture topologies. (a) Cloud-based centralized: remote data center aggregation, global coverage at high latency. (b) Edge/infrastructure-based: RSU local fusion, low latency with infrastructure dependency. (c) Decentralized P2P: on-board V2V fusion, high resilience at the cost of compute and synchronization overhead. The SDCN paradigm (Section 3.3 ) targets adaptive transitions among these topologies.
Table 4: Taxonomy of neural-communication co-design methods and cognitive synergy conditions.
Figure 5: Neural-communication co-design pipeline for collaborative perception. The pipeline transforms raw multi-agent observations into a collaborative perception output through five stages. (1) Multi-modal sensor observations from heterogeneous agents (LiDAR, camera, radar). (2) Neural feature encoding that compresses raw streams into compact latent representations. (3) An information bottleneck (IB) gate that enforces pragmatic communication by jointly minimizing transmitted redundancy while maximizing task-relevant mutual information given neighbors’ knowledge (Eq. 2 ). (4) A differentiable channel model (Deep JSCC) that directly maps latent codes to channel-modulated symbols, enabling end-to-end gradient flow from the task loss back through the physical layer and thereby avoiding the cliff effect. (5) Collaborative fusion that produces the multi-agent BEV detection, tracking, segmentation, and prediction output. The top feedback loop encodes the unified joint perception–communication Lagrangian (Eq. 3 ), enabling the entire stack to be optimized as a single differentiable system.
Table 5: Taxonomy of embodied synergy methods and cognitive synergy conditions.
Figure 6: The “three shadows” of distributed collaborative perception and their corresponding structural resilience mechanisms. The taxonomy addresses three foundational systemic vulnerabilities in collective intelligence: (1) Spatiotemporal Asynchrony (The Temporal Shadow): Communication latency and coordinate misalignments inherently induce catastrophic ghosting artifacts and localization degradation. This is mitigated through proactive neural manifold warping or predictive motion compensation networks that dynamically project stale historical features onto the current temporal baseline. (2) Epistemic Uncertainty (The Spatial Shadow): Localized line-of-sight occlusions (e.g., dynamic obstacles or blind spots) result in high-variance, uncertified feature volumes. The framework counteracts this by quantifying evidential uncertainty distributions and performing inverse-variance-weighted belief consensus to neutralize perceptual blind spots. (3) Semantic Scarcity (The Annotation Shadow): The high cost of multi-view cooperative annotation creates a severe training bottleneck. This is structurally resolved via the Dual Teacher-Student (CoDTS) paradigm, which distills mainstream and supplementary pseudo-labels from massive unlabeled data streams to achieve label-efficient collaborative optimization.
Table 6: Taxonomy of resilience methods and cognitive synergy conditions.
| Systemic Vulnerability | Root Cause | Key Mechanisms | Representative Methods | Open Challenges |
|---|---|---|---|---|
| Spatiotemporal Asynchrony | Communication latency, heterogeneous clocks, coordinate misalignment | Neural manifold warping (PNAT), latent motion compensation (LMC), motion-aware robust communication | V2X-ViT [ 23 ] , CoBEVT [ 24 ] , SyncNet [ 69 ] , MRCNet [ 97 ] , V2X-INCOP [ 98 ] | Real-time inference under ms latency budgets; formal alignment guarantees for safety-critical settings |
| Epistemic Uncertainty | Sensor degradation, adversarial agents, occlusion | Evidential deep learning (EDL), conformal prediction, inverse-variance Bayesian aggregation, Byzantine-robust fusion | MOT-CUP [ 99 ] , GevBEV [ 47 ] , ROBOSAC [ 96 ] , Krum/Median [ 106 ] | Provable Byzantine resilience with formal convergence guarantees; runtime uncertainty calibration on embedded hardware |
| Semantic Scarcity | High multi-view annotation cost, domain shift across deployments | Consensual contrastive learning (CCL), dual teacher-student (CoDTS), domain adversarial adaptation (DANN) | MA-SUE [ 100 ] , CoDTS [ 102 ] , domain generalization [ 110 ] | Closing the sim-to-real generalization gap; scaling to diverse real-world edge cases with minimal supervision |
Table 7: The “Three Shadows” of collaborative resilience: systemic vulnerabilities, mechanisms, and open challenges. Each shadow exposes a distinct failure surface in distributed perception; the corresponding mechanisms represent the current state of the art in addressing them.
Figure 7: Domain-specific synergy across collaborative-intelligence deployments. The illustration links four application contexts (industrial logistics, connected mobility, smart-city public services, and environmental monitoring) to the shared requirement for distributed sensing, communication-aware fusion, and proactive information acquisition.
Table 8: Taxonomy of domain-specific methods and cognitive synergy conditions.
Table 9: Taxonomy of trustworthy CI methods and cognitive synergy conditions. Methods are mapped onto the proposed five-dimensional framework.
| Technique | Protection Granularity | Comm. Overhead | Accuracy Penalty | Real-Time | Representative Work |
|---|---|---|---|---|---|
| Differential Privacy (DP) | Feature / trajectory | Low | Moderate | High | Feature-map DP [ 166 ] , trajectory DP [ 168 ] , DP-aware fusion [ 172 ] |
| Federated Learning (FL) | Model | Moderate | Low | Moderate | FedDrive [ 156 ] , FedCP [ 157 ] , Hierarchical FL [ 170 ] |
| Homomorphic Encryption (HE) | Raw data / feature | Very high | None | Low | Secure MKHE [ 158 ] |
| Secure Multi-Party Computation (SMPC) | Raw data / feature | High | None | Low–Mod. | Oblivious Fusion [ 155 ] , SMPC-based fusion [ 171 ] |
Table 10: Comparative overview of privacy-preserving techniques for collaborative perception. Each paradigm occupies a distinct position in the trade space between protection strength, communication overhead, and real-time feasibility.
Table 11: Taxonomy of open-world CI methods and cognitive synergy conditions.
Figure 8: Roadmap toward general collaborative intelligence. Four converging research trajectories, i.e., collaborative foundation models, pragmatic game-theoretic communication, hardware-agnostic universal semantics, and proactive swarm embodiment, connect the current state of collaborative perception with the long-term vision of GCI.
| Challenge | Severity | Scope | Near-Term Solvability | Representative Evidence | Potential Mitigation Pathways |
|---|---|---|---|---|---|
| Benchmark Saturation | High | V2X domain | Moderate | AP clustering within few percentage points [ 180 ] ; unreliable cross-paper comparisons [ 181 ] | Multi-dimensional metrics (accuracy–bandwidth–latency–energy); benchmarks with 10+ agents |
| Reproducibility Gap | High | All domains | Low | Missing code and incomplete training configurations; multi-agent training instability [ 183 ] | Standardized protocols [ 182 ] ; containerized implementations; locked test-set leaderboards |
| Sim-to-Real Gulf | Critical | All domains | Low | AWGN channel models vs. real mmWave [ 185 ] ; sensor fidelity gaps [ 184 ] ; MDrive planning-perception gap [ 179 ] | Domain randomization; hardware-in-the-loop; large-scale real-world multi-agent benchmarks |
| Under-Reported Failure Modes | Moderate | All domains | Moderate | LE/CE error taxonomy [ 181 ] ; Byzantine-robust fusion rarely benchmarked [ 106 ] ; latency-induced degradation [ 186 ] | Failure-analysis tracks; negative-result venues; standardized stress-test suites |
| Scalability & Agent Churn | High | Large-scale deploy. | Low | communication growth [ 187 ] ; cold-start problem unaddressed [ 190 ] | Hierarchical fusion [ 189 ] ; sparse attention [ 191 ] ; incremental onboarding protocols |
Table 12: Open challenges in collaborative perception: severity, scope, and mitigation pathways. Challenges are ordered approximately by their impact on the field’s transition from laboratory benchmarks to robust real-world deployment.
Figure 9: Overview of the GCI-Bench evaluation framework. (a) Scoring pipeline: raw operating points reported by a method on standard benchmarks are normalized into the five pillars of Table 13 , each scored on a 0–100 scale against the disclosed single-agent and oracle anchors, and aggregated into a composite under the explicit missing-data rule and maturity caps. (b) Case-study scorecard: pillar profiles of the five representative methods of Table 15 together with the aspirational GCI Target profile; unreported pillars are plotted at 0 only to close the polygons and denote missing rather than measured values.
| Pillar | What It Measures | Primary Metrics | Normalization | Benchmark Sources |
|---|---|---|---|---|
| P1. Perception Fidelity (PF) | Collaborative detection and tracking accuracy relative to single-agent baseline | [email protected], [email protected], NDS, AMOTA | = primary detection metric; oracle = perfect-communication upper bound | OPV2V [ 26 ] , V2XSet [ 23 ] , DAIR-V2X [ 32 ] , V2V4Real [ 117 ] |
| P2. Communication Efficiency (CE) | Perceptual gain per unit of communication cost | Bandwidth (KB/frame), latency (ms), bits per AP gain | , clamped to = communication cost; = fixed reference cost, so at | All benchmarks with communication measurements |
| P3. Resilience (RS) | Performance degradation under adverse conditions | AP drop under packet loss (10–50%), pose noise ( – m), adversarial perturbation | = relative AP drop under perturbation type ; averaged over types | OPV2V [ 26 ] , V2X-Sim [ 28 ] , custom stress-test suites [ 181 ] |
| P4. Scalability (SC) | Performance retention as the number of collaborating agents grows | AP at ; communication growth factor | , clamped to accuracy retention as the agent count grows from to | Benchmarks supporting 2–10 agents |
| P5. Trustworthiness (TW) | Privacy preservation, uncertainty calibration, and safety compliance | Privacy budget , expected calibration error (ECE), ISO 26262/21448 [ 159 , 160 ] compliance level | = domain-adjustable weights (summing to 1) | DAIR-V2X [ 32 ] , V2V4Real [ 117 ] , protocol-level tests |
Table 13: GCI-Bench five-pillar evaluation system. Each pillar is scored on a 0–100 scale and normalized against a single-agent baseline. A weighted aggregate yields the GCI Composite Score.
| Level | Designation | Score Range | Characteristics | Representative Archetype |
|---|---|---|---|---|
| L1 | Isolated | No effective collaboration; performance indistinguishable from or worse than single-agent baseline | Single-agent perception; naive late fusion that disregards neighbors’ raw features | |
| L2 | Reactive | – | Basic information sharing without adaptivity; fixed communication schedule | Broadcast-based intermediate fusion (V2VNet [ 33 ] , F-Cooper [ 39 ] ) |
| L3 | Adaptive | – | Communication-aware collaboration with selective information transmission | Saliency-driven fusion (Where2comm, Which2comm) |
| L4 | Proactive | – | Embodied collaboration closing the perception-action loop; proactive information foraging | MARL-driven embodied systems (CSAOT, Plan2comm, CoPnP) |
| L5 | Cognitive | – | Trustworthy, self-optimizing, cross-domain distributed cognitive system; plug-and-play interoperability across heterogeneous fleets | Aspirational target for GCI; no current system fully achieves this level |
Table 14: GCI maturity model. Five capability levels defined by composite score ranges, with representative method archetypes.
| Method | Stage | Comm. | PF | CE | RS | SC | TW | Comp. | [email protected] | KB/f | Lat. |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Late Fusion [ 32 ] | Late | Metadata | 25 | 99 | 20 | 75 | – | 55 (L2) † | 75 | Low | |
| V2VNet [ 33 ] | Intermed. | Broadcast | 62 | 59 | 30 | 25 | – | 44 (L3) | 82.4 | 26 | Med |
| V2X-ViT [ 23 ] | Intermed. | Broadcast | 66 | 6 | 35 | 30 | – | 34 (L2) | 88.2 ‡ | 60 | Med |
| Where2comm [ 21 ] | Intermed. | Saliency | 63 | 92 | 40 | 45 | – | 60 (L3) | 82.5 | 5 | Med |
| InfoCom [ 27 ] | Intermed. | IB-pruned | – | 98 | 35 | 50 | – | 61 (prov.) § | – | Med | |
| GCI Target | – | – | 95 | 90 | 90 | 85 | 85 | 89 (L5) | Low |
Table 15: GCI-Bench case study: pillar scores and reported operating points for five representative methods. Normalised pillar scores (PF/CE/RS/SC/TW, 0–100) are derived from the disclosed illustrative operating points under the protocol of Table 13 ; the three right-hand columns list the assumed raw operating points of the demonstration. Scores illustrate the framework rather than provide definitive rankings. “–” indicates that the original publication did not report data for that pillar.