cs.LGApr 4, 2026

Representational Collapse in Multi-Agent LLM Committees: Measurement and Diversity-Aware Consensus

Authors: Dipkumar Patel

Organizations: LLMs Research Inc.

Abstract

Multi-agent LLM committees replicate the same model under different role prompts and aggregate outputs by majority vote, implicitly assuming that agents contribute complementary evidence. We embed each agent's chain-of-thought rationale and measure pairwise similarity: across 100 GSM8K questions with three Qwen2.5-14B agents, mean cosine similarity is 0.888 and effective rank is 2.17 out of 3.0, a failure mode we term representational collapse. DALC, a training-free consensus protocol that computes diversity weights from embedding geometry, reaches 87% on GSM8K versus 84% for self-consistency at 26% lower token cost. Ablation experiments reveal 1-3 point per-protocol run-to-run variance, confirm that hint sharing contributes more than diversity weighting alone, and show that encoder choice strongly modulates collapse severity (cosine 0.908 with mxbai versus 0.888 with nomic) and downstream accuracy. The more robust finding is that collapse is measurable, worsens on harder tasks, and that the choice of embedding proxy is a first-order design decision for any latent communication protocol.

Figures & tables

Explore similar work

CardsList
  1. Collective Regimes in Multi-Agent LLMs under Reasoning Effort and Communication Topology

    Sep 27, 2026Machiko Hirota, Akshara Nadayanur Sathis Kanna, Ujwal Kumar +1Multi-Agent LLM SystemsMulti-Agent Coordination

  2. Prompted Identity Degrades Cooperation in Multi-Agent LLM Systems

    Sep 28, 2026Xavier Del Giudice, Alessio Palma, Matteo Migliarini +2Multi-Agent LLM SystemsMulti-Agent Coordination

  3. Drivers of LLM Consensus Formation Under Competing Incentives: A Simulation Study

    Oct 14, 2026Isabel Jones, Matthew Edwards, Janina A. HoffmannMulti-Agent ConsensusMulti-Agent LLM Systems