Behavioral Probes for Information Flow in LLM Swarms
Authors: Junhui Chen, Yu Zhang, Lantian Li, Dong Wang
Organizations: School of Artificial Intelligence, Beijing University of Posts and Telecommunications, China · Center for Speech and Language Technologies, BNRist, Tsinghua University, China
Most existing approaches to LLM swarm optimization are utility-driven, focusing on final task utility while leaving the processing, integration, and reliance of information within individual interactions largely implicit. To address this, we study LLM swarm interactions from an information-flow-based view and introduce behavioral probes to quantify information flow patterns. Specifically, we repurpose External Context Score (ECS) and Parametric Knowledge Score (PKS)---originally designed for hallucination analysis---to measure a node's external-context dependence and parametric-knowledge dependence, respectively. Across controlled and realistic swarm settings, we observe consistent patterns: ECS increases with input relevance and distinguishes useful multi-source contributions, while PKS decreases as informative context accumulates. Building on this heuristic interpretation, we demonstrate how these probes can complement utility-driven structure optimization in two controlled applications. Probe-guided node pruning reduces redundant communication and token consumption, and probe-guided reasoning-path search accelerates convergence compared to utility-only baselines. Our empirical results suggest that behavioral probes can make the internal information flow of LLM swarms observable and serve as effective auxiliary signals for structure optimization.
Figures & tables
Figure 1: Node-level response patterns under four context conditions: irrelevant ( bubble ), related but unhelpful ( rel ), oracle denotes inverse relational evidence that is sufficiently informative for reverse inference, and answer denotes context directly containing the answer. Curves and translucent bars show score densities, dashed vertical lines mark group means, and the annotation boxes report mean and standard deviation. Overall, ECS increases with context relevance, while PKS decreases when the answer can be inferred from the provided context.
Figure 2: Behavioral probes in multi-source aggregation and chain-reasoning settings. (a) Z-standardized segment-level ECS of the Critic with respect to upstream expert outputs; higher values indicate stronger reliance on the corresponding segment. (b) PKS trajectories along the reasoning chain, with labels indicating the last taxonomic level that introduces new information. Solid lines show means and shaded regions show the 15–85 percentile range. Overall, seg-ECS highlights domain-relevant upstream sources, while PKS decreases only when informative context continues to accumulate.
Figure 3: Behavioral probes in a realistic tool-augmented GAIA pipeline. Panel (a) compares ECS for decision-critical and non-critical tool evidence across WebSearch and FileAnalyse ; panel (b) compares ECS for function-calling search-result segments labeled as useful or irrelevant by the corresponding WebSearch analysis output; panel (c) compares CombineAnswer PKS before and after removing WebSearch evidence.
Model
Signal
Method
Weights
Efficiency
DA
LA
BA
Rdn
Acc (%) ↑
Tok/Q ↓
Qwen-2.5-32B
Vanilla utility
GPT Swarm
1.515
1.505
1.438
2.943
80.6
2294
Agent Net
1.876
1.834
1.849
3.683
80.8
2744
Agent Prune
1.140
0.837
0.708
1.545
80.8
1879
Probe-guided utility
Probe-ECS ∗
1.859
1.650
0.551
2.201
79.4
2345
Probe-PKS ∗
1.052
0.432
1.599
2.031
79.4
1710
Table 1: Node-pruning results across baselines. DA/LA/BA denote cumulative inbound weights to the final decision node; Rdn sums redundant LA/BA weights; Tok/Q is average tokens per question. The Signal column distinguishes utility-driven baselines from probe-guided utility strategies. ∗ marks variants using ECS–PKS behavioral profiles.
Figure 4: Critical edge weight trends under three warm-starts. After 50 utility-only warm-up iterations, the PKS-based convergence probe is introduced. The panels show good, moderate, and poor warm-starts, defined by the critical-edge weight before probe intervention. Solid lines denote the mean weight over critical edges, with shaded regions showing the 15–85 percentile range. Across all cases, the probe accelerates critical-edge growth compared with the utility-only baseline.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Field
Text
Question
Who is Kate Mara’s father?
bubble
The rural landscape bloomed with vibrant wildflowers, each petal soaking up the sun’s warmth. Farmers tended to their fields, hopeful for a bountiful harvest under the clear blue sky.
rel
Kate Mara is an American actress renowned for roles in film and television, including “House of Cards” and “Captain America: The First Avenger.” She is also recognized for her fashion industry involvement and philanthropic work.
oracle
Kate Mara is Timothy Christopher Mara’s Child.
answer
Timothy Christopher Mara is Kate Mara’s father.
Appendix
Table 2: Example context conditions for the node-level response experiment.
Metric
Mean adj. δ
95% CI
Spearman ρ
p
ECS
0.624
[0.610, 0.640]
0.857
<2×10−4
PKS
-0.157
[-0.170, -0.143]
-0.308
<2×10−4
Appendix
Table 3: Statistical summary for the node-level response experiment in Figure 1 . Mean adjacent Cliff’s δ averages the three ordered comparisons bubble → rel , rel → oracle , and oracle → answer . Confidence intervals are 95% bootstrap intervals, and p -values are from paired permutation Spearman tests.
Topic Group
Included Subjects
Math
abstract algebra, college mathematics, elementary mathematics, high school mathematics, high school statistics, formal logic
Computer Science
college computer science, high school computer science, computer security, machine learning, electrical engineering
Physics
college physics, conceptual physics, high school physics, astronomy
Chemistry / Biology / Medicine
anatomy, clinical knowledge, college biology, college chemistry, college medicine, high school biology, high school chemistry, human aging, human sexuality, medical genetics, nutrition, professional medicine, virology
Law / Politics / History
high school US history, high school world history, high school European history, high school government and politics, international law, jurisprudence, professional law, security studies, US foreign policy, prehistory
Business / Economics
business ethics, econometrics, global facts, high school macroeconomics, high school microeconomics, management, marketing, professional accounting, public relations
Appendix
Table 4: Topic grouping of MMLU subjects used in the expert-specialization analysis. The first column gives the aggregated topic group used on figure axes, and the second column lists the original MMLU subjects assigned to that group. This grouping enables cross-domain comparison of expert-conditioned ECS responses.
Table 5: Statistical summary for the GAIA tool-augmented reasoning experiment in Figure 3 . Confidence intervals are 95% bootstrap intervals for mean differences. Positive differences indicate larger ECS for critical RAG evidence in panel (a), larger ECS for relevant WebSearch segments in panel (b), and larger PKS after removing WebSearch evidence in panel (c).
Figure 5: ECS–PKS behavioral profiles of functional categories in the GAIA dataset. The x-axis is PKS, the y-axis is ECS, colors denote functional log categories, and black crosses mark category centers. The profile clusters align with functional roles, supporting a task-dependent region Ωtask for probe-guided pruning.