Seeing the Forest for the Trees: Emergent Agreement from Individual Conversations

Adi Simhi; Yaniv Nikankin; Jonathan Herzig; Idan Szpektor; Yonatan Belinkov; Shay B. Cohen

cs.CLOct 21, 2026

Seeing the Forest for the Trees: Emergent Agreement from Individual Conversations

Authors: Adi Simhi, Yaniv Nikankin, Jonathan Herzig, Idan Szpektor, Yonatan Belinkov, Shay B. Cohen

Organizations: Technion – Israel Institute of Technology · Google Research · University of Edinburgh

Abstract

As LLM agents interact in the wild and shape each other's decisions, understanding how they converge toward agreement becomes important. We study this in multi-turn conversations where two LLMs discuss subjective matters, can agree or disagree, and may change their stances. Using a linear classifier on individual conversation activations, we identify an internal representation of agreement. We find that it varies substantially across turns and conversations and does not reliably track external agreement at the individual conversation level. We therefore move beyond the representation of agreement in individual conversations, by aggregating the agreement representation across conversations. Externally, this aggregate view reveals a simple structure: when pooled across conversations, agreement can be predicted well with a Markov process, with agreement at each turn being a direct function of agreement at the previous turn. Similarly, when internal agreement is aggregated, it couples with the external Markov agreement through a power-law mapping, exposing Markovian dynamics in internal representations. Additionally, we find that after a change of stance, models express agreement externally before their internal representations reflect the same shift. These results reveal a joint structure between internal and external agreement that emerges only when internal features are aggregated across conversations. Code available at https://github.com/technion-cs-nlp/SeeingTheForestForTheTrees.

Figures & tables

Figure 1: Overview. We analyze how models converge to agreement in multi-agent conversational scenarios. (a) In an example conversation, two models are conditioned on opposing personas. They discuss a scenario, turn by turn, until one changes its opinion and they converge to agreement. (b) Regressing the projection of agreement in model activations of individual conversations yields a measure with high variance. It varies both between conversations and between turns. (c) When conversations are aggregated, the agreement probability at any turn can be predicted with a first-order Markov process. (d) When aggregated, the internal agreement measure is related to behavioral Markov agreement through a power law relation. This internal curve exemplifies a new method of observing model internal concepts.

Figure 2: Internal agreement has high variance. For each model–dataset pair, we randomly sample ten conversations and plot internal agreement

\smash{z^{(c,i)}_{t}}

across turns. The results complement Table 1 and show that internal agreement varies substantially and does not correlate with behavioral convergence to agreement.

		Dataset
Model	Metric	Beh-Disp.	Global	Manager	Self-Coord.	Wealthseeking
Ministral	Across	0.97 $\pm$ 0.14	0.93 $\pm$ 0.12	0.95 $\pm$ 0.07	0.96 $\pm$ 0.09	0.96 $\pm$ 0.05
Ministral	Within	0.71 $\pm$ 0.18	0.64 $\pm$ 0.21	0.66 $\pm$ 0.18	0.63 $\pm$ 0.21	0.65 $\pm$ 0.17
Qwen	Across	0.96 $\pm$ 0.16	0.99 $\pm$ 0.08	0.90 $\pm$ 0.07	0.99 $\pm$ 0.04	0.95 $\pm$ 0.11
Qwen	Within	0.60 $\pm$ 0.19	0.62 $\pm$ 0.19	0.59 $\pm$ 0.19	0.61 $\pm$ 0.18	0.58 $\pm$ 0.20
Gemma	Across	0.94 $\pm$ 0.15	0.99 $\pm$ 0.27	0.95 $\pm$ 0.29	0.98 $\pm$ 0.25	0.99 $\pm$ 0.23

Table 1: Variance measures of internal agreement

\bm{z^{(c,i)}_{t}}

. Variance across conversations is near the upper limit (

1.0

), indicating that internal agreement varies greatly across conversations. Variance within conversations is above

0.5

in all setups, indicating that internal agreement changes direction between turns for at least half the conversation turns, on average.

Figure 3: Behavioral agreement probability can be predicted with a first-order Markov model. For each model–dataset pair, we plot the empirical probability of agreement at each turn. We also plot three predictors for this probability: a first-order Markov model, a second-order Markov model, and a unigram baseline (constant probability across turns). Across setups, the first-order Markov model closely predicts the empirical ground truth.

Figure 4: Aggregated internal agreement is coupled to behavioral Markov probability via a power-law relation. We fit a power-law to map behavioral agreement probabilities to aggregated internal agreement for each model–dataset pair, and plot it on a held-out set of conversations. Showing that

\bar{z}(t)

and

p(t)

are coupled via a power law.

	Dataset
Model	Beh-Disp.	Global	Manager	Self-Coord.	Wealthseeking
Ministral	0.55	0.54	0.88	0.75	0.64
Qwen	0.30	0.19	0.67	0.11	0.87
Gemma	0.92	0.35	0.82	0.79	0.55

Table 2:

\bm{\mathrm{NR}^{2}}

results between aggregated internal agreement (

\bm{\bar{z}(t)}

) and behavioral Markov agreement (

\bm{p(t)}

). Higher values indicate a better match, with scores over

0.5

meaning that the behavioral Markov agreement captures the shape of the internal agreement trajectory rather than just predicting its mean level. The scores are high for all non-degenerate model–dataset pairs. This suggests that behavioral and internal agreement are generally linked at the population level.

Figure 5: The power-law coupling is missing with a random internal direction. As a control, we fit a power-law to map behavioral Markov agreement to a random aggregated internal direction, confirming that the relation in Figure 4 is not trivially caused by aggregation.

Figure 6: Distinguishability of internal agreement from reference agreement. Each cell shows $\Delta_{\ell}$ between the post-opinion-change representation set ( $Z_{\ell}$ ) and the reference agreement set ( $Z_{\text{ref}}$ ). Values near $0$ indicate indistinguishable sets, while larger values indicate greater separation. Internal representations remain distinguishable from reference agreement after an opinion change, and slowly converge towards it over subsequent turns, showing that internal agreement lags behind behavioral agreement.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Start Disagree	Finish Disagree	Missing Label	Co-opinion change	Opinion change $\geq$ 1	Opinion change Count
Ministral	42.5% $\pm$ 16.3%	18.4% $\pm$ 3.0%	13.3% $\pm$ 5.0%	48.3% $\pm$ 9.4%	99.6% $\pm$ 0.5%	2.48 $\pm$ 0.35
Qwen	42.6% $\pm$ 14.5%	25.1% $\pm$ 11.2%	0.0% $\pm$ 0.0%	16.0% $\pm$ 8.2%	90.5% $\pm$ 6.1%	1.25 $\pm$ 0.23
Gemma	69.2% $\pm$ 19.1%	60.9% $\pm$ 22.2%	0.1% $\pm$ 0.1%	0.5% $\pm$ 0.8%	41.1% $\pm$ 23.4%	0.26 $\pm$ 0.17
Gemini	68.7% $\pm$ 25.9%	22.3% $\pm$ 17.4%	0.0% $\pm$ 0.0%	8.3% $\pm$ 5.9%	88.7% $\pm$ 10.3%	0.96 $\pm$ 0.20

Appendix

Table 3: Per-model conversation statistics across datasets. Start Disagree is the percentage of scenarios that start with disagreement. Finish Disagree is the percentage of scenarios that end in disagreement. Missing Label is the percentage of scenarios containing at least one unparsable label. Co-opinion change is the percentage of scenarios containing at least one turn where both models change their labels simultaneously. Opinion change

\geq 1

is the percentage of scenarios with at least one label change by either model. Opinion change count is the mean number of label changes per model instance per scenario.

Figure 7: Per-turn agreement direction, embedded on a unit circle. The circular embeddings become more similar after the first few turns, indicating that the agreement directions converge as the conversation progresses.

		Dataset
Model	Metric	Beh-Disp.	Global	Manager	Self-Coord.	Wealthseeking
Ministral	Across	0.97 $\pm$ 0.12	0.93 $\pm$ 0.13	0.94 $\pm$ 0.10	0.95 $\pm$ 0.11	0.96 $\pm$ 0.08
Ministral	Within	0.63 $\pm$ 0.20	0.65 $\pm$ 0.17	0.63 $\pm$ 0.21	0.61 $\pm$ 0.22	0.66 $\pm$ 0.17
Qwen	Across	0.96 $\pm$ 0.14	1.00 $\pm$ 0.12	0.93 $\pm$ 0.13	1.00 $\pm$ 0.11	0.97 $\pm$ 0.19
Qwen	Within	0.59 $\pm$ 0.15	0.59 $\pm$ 0.17	0.59 $\pm$ 0.22	0.61 $\pm$ 0.19	0.60 $\pm$ 0.18
Gemma	Across	0.90 $\pm$ 0.24	1.00 $\pm$ 0.25	0.95 $\pm$ 0.30	0.98 $\pm$ 0.31	0.99 $\pm$ 0.34

Appendix

Table 4: Conversation variance using logistic regression. The results are similar to those obtained with LDA in Table 1 .

Figure 8: Agreement probability can be predicted with a Markov process. For Gemini per dataset, we plot the empirical probability of agreement at each turn. We also plot three predictors for this probability: a first-order Markov model, a second-order Markov model, and a unigram baseline (constant probability across turns). Across setups, the first-order Markov model closely predicts the empirical ground truth.

		Dataset
Model	Param	Beh-Disp.	Global	Manager	Self-Coord.	Wealthseeking
Ministral	$\alpha$	0.407	4.572	1.487	11.861	6.967
	$\beta$	0.273	-0.643	-0.042	-0.609	-0.644
	$\gamma$	1.602	8.845	4.644	10.000	10.000
Qwen	$\alpha$	0.754	0.809	2.069	1.197	1.623
	$\beta$	-0.087	-0.267	-0.226	-0.218	-0.251

Appendix

Table 5: Fitted power-law parameters for the relationship between Markov-predicted behavioral agreement and internal agreement in Section 5.1 .

Figure 9: A random internal direction does not track behavioral Markov agreement even after aggregation.

Figure 10: Relationship between aggregated internal agreement and behavioral Markov-predicted agreement at 30% relative depth.

Figure 11: Relationship between aggregated internal agreement and behavioral Markov-predicted agreement at 50% relative depth.

Figure 12: Relationship between aggregated internal agreement and behavioral Markov-predicted agreement at 100% relative depth.

Figure 13: Relationship between aggregated internal agreement at the first answer token and behavioral Markov agreement.

Figure 14: Relationship between aggregated internal agreement and behavioral Markov agreement using internal directions identified with logistic regression.

Figure 15: Internal agreement trajectories form at most two subgroups. For each model–dataset pair, we cluster the per-conversation change in internal agreement relative to the first turn (

\Delta z

) with

k

-means and plot the mean trajectory of each cluster across turns. Most pairs show a single increasing trajectory. In the remaining pairs, mostly in Gemma, one subgroup increases in agreement while the other stays close to zero.

Figure 16: Distinguishability from agreement using the first turn agreement direction: Internal population convergence lags behavioral convergence. Comparing post-opinion change representations against the golden “agree” reference; Close to $0$ denotes indistinguishable from agreement and $1$ maximally distinguishable.

Figure 17: Hedging phrases following an opinion change. Qwen and Gemma use more hedging phrases after an opinion change than in other turns.

Multi-Agent Consensus

Multi-agent consensus studies how interacting agents reach shared decisions or states across networks, and how to make agreement reliable.

See all 126 papers

Silent Dissent: LLM Agents That Yield to the Majority Still Represent Their Original Premise
Oct 2, 2026Ziang Ni, Peng ZouNormative Conformity in Language Models Multi-Agent Debate
Sun Yat-sen University Guangzhou, China
Multi-agent discussion gains less when dissent is withheld
Sep 29, 2026Chand Sahil Mansuri, Xin Wang, Mengying Li +4Multi-Agent LLM Systems Opinion Dynamics
School of Systems Science and Industrial Engineering, Binghamton University, Binghamton, NY, USA · School of Management, Binghamton University, Binghamton, NY, USA · IBM T. J. Watson Research Center, Yorktown Heights, NY, USA
Drivers of LLM Consensus Formation Under Competing Incentives: A Simulation Study
Oct 14, 2026Isabel Jones, Matthew Edwards, Janina A. HoffmannMulti-Agent Consensus Multi-Agent LLM Systems
University of Bristol Bristol, United Kingdom · University of Bath Bath, United Kingdom

Oct 2, 2026J/K move · Enter open · S save

Silent Dissent: LLM Agents That Yield to the Majority Still Represent Their Original Premise

Ziang Ni, Peng Zou

Sun Yat-sen University Guangzhou, China

Normative Conformity in Language Models Multi-Agent Debate Reasoning Consistency in Language Models Multi-Agent Consensus Multi-Agent Deliberation

Open paper AskPDF arXiv

Multi-agent debate is increasingly used to reach consensus among LLM agents, yet agents often yield to a unanimous majority. When an agent changes its answer, has it changed its mind or only its statement? We study this with two-hop factual questions whose intermediate entity (the bridge, e.g. the country in "the capital of the country where the Sagrada Familia is located") is never stated by anyone. Scripted peers, in the role of Asch's confederates, unanimously assert a wrong answer taken from another fact with a different bridge. At the moment the agent answers, we read the bridge from its residual stream with the Jacobian lens (J-lens) and, for comparison, the logit lens. In pre-registered tests on held-out facts with four open-weight models, agents of Qwen3.5-4B, Qwen3.6-27B and Gemma-4-E4B-it that gave in still represented their original bridge in the pre-registered layers below the output (hit@100 above a control entity: 0.85, 0.22 and 0.24), where the logit lens rarely ranked it among the top 100 tokens (0.00-0.06). These agents also represented the bridge behind the peers' answer, beyond a mention baseline. A pre-registered addendum hid the agent's earlier answer or removed it: agents that gave in still represented their original bridge in all four models (0.43, 0.29, 0.37 and 0.25 with the answer hidden), including Llama-3.1-8B-Instruct, which barely did so with its answer in view (0.03). The premise can thus be computed from the question alone while the agent states the majority's answer. Hiding the earlier answer also changed conformity: Qwen3.5-4B gave in on 89% of questions instead of 8%. In exploratory interventions, injecting the bridge's J-lens direction brought agents back to their original answer only in the two Qwen models. Stated consensus in multi-agent debate can thus overstate agreement. We also report the negative results of our pre-registered program.

Abstract

Multi-Agent Consensus

Explore similar work

Silent Dissent: LLM Agents That Yield to the Majority Still Represent Their Original Premise

Multi-agent discussion gains less when dissent is withheld

Drivers of LLM Consensus Formation Under Competing Incentives: A Simulation Study