We introduce CONCORD, a privacy-aware asynchronous assistant-to-assistant (A2A) framework that leverages collaboration between proactive speech-based AI. As agents evolve from reactive to always-listening assistants, they face a core privacy risk (of capturing non-consenting speakers), which makes their social deployment a challenge. To overcome this, we implement CONCORD, which enforces owner-only speech capture via real-time speaker verification, producing a one-sided transcript that incurs missing context but preserves privacy. We demonstrate that CONCORD can safely recover necessary context through (1) spatio-temporal context resolution, (2) information gap detection, and (3) minimal A2A queries governed by a relationship-aware disclosure. Instead of hallucination-prone inferring, CONCORD treats context recovery as a negotiated safe exchange between assistants. Across a multi-domain dialogue dataset, CONCORD achieves 91.4% recall in gap detection, 96% relationship classification accuracy, and 97% true negative rate in privacy-sensitive disclosure decisions. By reframing always-listening AI as a coordination problem between privacy-preserving agents, CONCORD offers a practical path toward socially deployable proactive conversational agents.
Figures & tables
Figure 1: Concord : one-sided transcript processing pipeline. (Left) Multi-turn two-user dialogues, where each user’s assistant only records their authenticated user. (Right) The system performs spatio-temporal reference resolution and identifies information gaps ( 2.2 ), then applies a decision filter to approve queries based on the relationship ( 2.3 ).
Relation Level
Proximity
Linguistic Indicators
Disclosure Policy by Sensitivity
Low
Mid
High
L1
Intimate (High)
Implicit references to private spaces ( the bedroom ); elliptical commands ( grab that ); relational nouns ( mom , babe ); high ambiguity tolerance due to shared grounding.
Direct Reveal
Direct Reveal
Direct Reveal
L2
Social (Moderate)
Balanced shared history; first-name address; conversational but more explicit entity references; moderate formality.
Direct Reveal
Approval Loop
Suppress
L3
Professional (Low)
Formal register; honorifics ( Dr., Mr., Ms. ); task-oriented vocabulary; distancing modal verbs ( could you, would you ).
Table 2: Dataset statistics for the Concord framework. High-st.: High-stakes and Act.: Actionable Items.
Framework Component
Evaluation Metric
Zero-shot LLM
SOTA (CoT-SR)
Concord
Local Reference Resolution
True Positive Rate (TPR % ↑ )
28.4
35.49
78.3
Sentence Similarity (% ↑ )
34.9
65.7
58.1
Info Gap Detection
True Positive Rate (TPR % ↑ )
46.2
45.8
91.4
False Positive Rate (FPR % ↓ )
22.8
19.5
6.8
Relationship Assessment
Classification Accuracy (% ↑ )
95
97.0
96.4
Hybrid Decision Gate
True Negative Rate (TNR % ↑ )
66.5
94
97.0
Table 3: System-level performance across the different components of Concord . Our approach achieves high Safety Integrity (97.0% TNR) while maintaining high utility for proactive gap resolution.