Robots operating in the real world are inherently subject to partial observability, particularly in multi-robot collaboration, where individual observations may be insufficient for effective task execution. A key challenge is leveraging complementary observations from partner robots to support collaborative decision-making. We argue that information sharing should go beyond passively transmitting raw observations or generic visual features. Instead, robots should selectively encode task-relevant information according to task objectives and execution progress. To this end, we propose a task-oriented active context sharing framework that transforms robot observations into compact representations, preserving critical information under constrained communication budgets. The framework adopts a two-stage training paradigm. In the first stage, onboard observations and task instructions are jointly encoded into compact context representations. Task-oriented future prediction and soft task-progress correspondence learning guide this collaboration-agnostic compression, preserving information about task objectives and execution progress. In the second stage, the learned representations are integrated into a pretrained Vision-Language-Action (VLA) policy, enabling bidirectional context exchange. Each robot combines complementary partner information with local observations to guide action generation. Action supervision jointly refines the context representations and policy, aligning information sharing with collaborative decision-making. Controlled experiments demonstrate substantial performance gains under constrained communication budgets, while three real-world experiments validate the framework in complex dual-robot tasks. These results highlight a shift from passive visual transmission toward active, task-oriented context sharing that directly supports collaborative decision-making.
Figures & tables
Figure 1: Overview of the two-stage active context sharing framework. In Stage 1, multi-camera observations and task instructions are compressed into shared context, optimized through weighted future visual feature prediction and soft-label contrastive learning. In Stage 2, robots bidirectionally exchange local context and condition their VLA policies on shared context to generate continuous actions for collaborative decision-making.
Figure 2: Soft Task-Progress Correspondence Learning. For each observation, augmented views are encoded by the task-conditioned context encoder and treated as positive pairs. Task- and progress-based affinities define the similarity matrix for contrastive learning, promoting cross-view consistency, inter-task discriminability, and intra-task temporal continuity.
Figure 3: Representative rollouts of designed collaborative tasks. Keyframes highlight critical moments of inter-robot context sharing during task execution.
Table 5: Context token budget vs. task success rate.
Figure 4: Representative rollouts of more collaborative tasks. Keyframes highlight critical moments of inter-robot context sharing during task execution.
Figure 5: Paired visual attention maps of Robots A and B during refrigerator restocking at four key moments. Each column shows the two robots’ main-camera observations at the same timestep, with brighter regions indicating higher attention scores.