Thought Blocks: Non-Autoregressive Latent Communication for Multi-Agent Systems via Trajectory-to-State Compilation

Zhenhan Guan; Xiaolong Liu; Hengwei Ye; Zheng Tian

cs.MAOct 16, 2026

Thought Blocks: Non-Autoregressive Latent Communication for Multi-Agent Systems via Trajectory-to-State Compilation

Authors: Zhenhan Guan, Xiaolong Liu, Hengwei Ye, Zheng Tian

Organizations: ShanghaiTech University, Shanghai, China · AI Lab, The Yangtze River Delta, Changzhou, China

Abstract

LLM-based multi-agent systems typically communicate through text, incurring repeated generation and re-encoding costs. Although latent communication removes explicit verbalization, constructing latent messages may still require autoregressive latent trajectories. We therefore distinguish message representation from message construction and ask whether agents can directly construct useful continuation states without executing the trajectories that normally produce them. We formulate this problem as trajectory-to-state compilation and introduce Thought Blocks, fixed-size blocks of latent key--value (KV) continuation memory constructed in one shot. Thought Blocks achieve a 2.5 times average latency speedup over autoregressive latent communication. Compared with text-based multi-agent communication, Thought Blocks reduce text tokens by 45.7% and autoregressive steps by 84.8%, achieving an 8.6 times average latency speedup. These results demonstrate that trajectory-to-state compilation enables efficient inter-agent latent communication without sequential latent rollouts at inference.

Figures & tables

Method	Inter-agent communication	KV message	Sequential latent rollout	Iterative latent refinement	Direct post-trajectory state
Single-agent latent reasoning
Coconut Hao et al. (2024)	✗	✗	✓	✗	✗
PCCoT Wu et al. (2025)	✗	✗	✗	✓	✗
CoT2 Gozeten et al. (2026)	✗	✗	✓	✗	✗
LaDiR Kang et al. (2025)	✗	✗	✗	✓	✗
Trajectory-produced latent communication

Table 1: Comparison of latent reasoning and communication mechanisms.

		Qwen2.5-7B					Qwen3-8B
Task	Method	Acc.	Text Tokens	Passes ∗	AR Steps †	Latency †	Acc.	Text Tokens	Passes ∗	AR Steps †	Latency †
Reasoning	Single Text	69.06	233	-	53	2.18	86.45	233	-	50	1.75
	Text-MAS	74.91	1717	483	492	16.09	89.75	1546	375	380	16.80
	Latent-MAS	73.85	751	12	30	3.85	89.62	752	12	16	3.99
	Ours	73.77	737	3	6	0.85	89.24	738	3	10	1.23
	Improve	-1.13	57.1%	99.4%	98.7%	18.88 $\times$	-0.51	52.3%	99.2%	97.3%	13.66 $\times$

Table 2: Main comparison across four LLM backbones and three task categories. Results are category means. Improve compares Ours with Text-MAS: accuracy is Ours

-

Text-MAS, efficiency metrics are relative reductions, and latency is speedup. Bold indicates the best multi-agent result.

Method	ARC-C	ARC-E	BoolQ	GPQA	GSM8K	MedQA(US)	MedQA(TW)	MedQA(ML)	MMLU	OpenBookQA	QuAIL
Ours	91.76	96.91	90.25	43.89	82.05	71.16	73.41	80.12	82.79	86.23	77.44
Zero $E$	83.00	84.96	83.89	32.54	70.40	68.01	71.72	79.22	80.09	82.71	71.25
Random $E$	80.70	83.25	79.52	30.37	70.56	66.35	68.81	76.43	77.61	78.74	69.28
$R-only$	79.11	78.63	71.39	15.31	58.94	54.66	57.98	68.25	65.72	64.43	59.67
$C-only$	82.63	82.94	76.08	19.15	66.36	60.32	64.57	75.61	72.43	73.65	64.98

Table 3: Ablation study of our method and its variants on the datasets. Accuracy (%) is reported.

Task Category	Metric	B=1	B=4	B=8	B=12	B=16	B=24	B=32
Reasoning	Accuracy	87.14	89.24	89.46	89.69	89.35	89.34	90.02
	Latency (s)	1.13	1.23	1.26	1.27	1.21	1.25	1.29
	KV Load (MB)	4.66	5.71	7.12	8.52	9.93	12.74	15.55
Question Answering	Accuracy	65.49	70.27	71.24	71.23	71.66	71.82	72.21
	Latency (s)	3.12	3.20	3.23	3.27	3.23	3.20	3.34
	KV Load (MB)	7.37	8.42	9.83	11.23	12.64	15.45	18.26

Table 4: Effect of Thought Block size on Qwen3-8B.

Method	ARC-C	ARC-E	BoolQ	GPQA	GSM8K	MedQA(ML)	MedQA(TW)	MedQA(US)	MMLU	OpenBookQA	QuAIL
Ours	6.25	5.66	8.61	10.13	5.61	6.78	7.28	10.76	7.17	5.33	18.26
Latent-MAS	41.96	39.95	59.48	71.42	38.99	47.08	51.14	75.68	52.87	36.42	126.18
Text-MAS	58.02	55.49	64.83	93.86	50.43	67.54	75.49	89.27	65.75	52.79	108.91

Table 5: Loaded KV-cache size (MB) during decoding on Qwen3-8B.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Figure A.1: Overview of our Thought Block framework for latent multi-agent communication. Each agent first encodes its local context together with the compressed communication history, where earlier interactions are retained through Beacon KV states while the most recent context preserves its raw-token states. During training, a frozen Beacon teacher autoregressively constructs a latent continuation trajectory, from which a fixed number of model-native continuation states are extracted to form the supervision target. The Thought Block Writer is then trained to predict these continuation-memory states directly from the pre-trajectory representation. At inference time, the teacher trajectory is removed entirely: the Writer generates the Thought Block in a single forward pass, and the resulting latent states are directly propagated to subsequent agents through the KV cache without decoding intermediate textual messages. This design preserves model-native latent information while substantially reducing the autoregressive communication overhead of conventional text-based multi-agent systems.

Dataset	Model	Method	Acc.	Text Tokens	Comm. Passes	AR Steps	Latency
ARC-C	Qwen2.5-7B	Ours	82.38	784	3	4	0.68
		Latent-MAS	83.39	788	12	17	3.39
		Text-MAS	83.96	1757	479	482	13.36
		Single	81.19	199	-	5	0.09
		Improve	-1.58	55.4%	99.4%	99.6%	19.6 $\times$
	Qwen3-8B	Ours	91.76	769	3	4	0.61

Appendix

Table B.1: Main performance comparison on all models across benchmarks. For accuracy, Improve is computed as Ours

-

Text-MAS. For efficiency metrics, Improve reports the reduction or speedup of Ours relative to Text-MAS. The best and second-best results among Ours, Latent-MAS, and Text-MAS are highlighted in bold and underlined, respectively. For Improve rows with an accuracy difference greater than

-1.5

points. (Part 1/6: ARC-C, ARC-E.)

Dataset	Model	Method	Acc.	Text Tokens	Comm. Passes	AR Steps	Latency
BoolQ	Qwen2.5-7B	Ours	84.19	1118	3	5	1.49
		Latent-MAS	86.76	1119	12	16	3.35
		Text-MAS	85.85	1938	400	404	13.92
		Single	83.73	280	-	6	0.16
		Improve	-1.66	42.3%	99.2%	99.3%	9.3 $\times$
	Qwen3-8B	Ours	90.25	1121	3	6	0.68

Appendix

Table B.2: Main performance comparison on all models across benchmarks (continued; Part 2/6: BoolQ, GPQA).

Dataset	Model	Method	Acc.	Text Tokens	Comm. Passes	AR Steps	Latency
GSM8K	Qwen2.5-7B	Ours	44.05	740	3	14	0.91
		Latent-MAS	44.87	787	12	73	6.34
		Text-MAS	43.06	1823	530	552	18.60
		Single	31.84	372	-	195	8.26
		Improve	+0.99	59.4%	99.4%	97.5%	20.4 $\times$
	Qwen3-8B	Ours	82.05	755	3	27	2.41

Appendix

Table B.3: Main performance comparison on all models across benchmarks (continued; Part 3/6: GSM8K, MedQA(ML)).

Dataset	Model	Method	Acc.	Text Tokens	Comm. Passes	AR Steps	Latency
MedQA(TW)	Qwen2.5-7B	Ours	60.20	960	3	5	0.86
		Latent-MAS	59.46	971	12	26	3.91
		Text-MAS	62.88	2092	565	580	19.85
		Single	57.64	247	-	13	0.35
		Improve	-2.68	54.1%	99.5%	99.5%	23.1 $\times$
	Qwen3-8B	Ours	73.41	989	3	8	2.34

Appendix

Table B.4: Main performance comparison on all models across benchmarks (continued; Part 4/6: MedQA(TW), MedQA(US)).

Dataset	Model	Method	Acc.	Text Tokens	Comm. Passes	AR Steps	Latency
OpenBooKQA	Qwen2.5-7B	Ours	77.60	679	3	4	0.84
		Latent-MAS	75.80	681	12	16	3.24
		Text-MAS	80.40	1646	479	486	17.02
		Single	71.42	176	-	7	0.23
		Improve	-2.80	58.7%	99.4%	99.6%	20.3 $\times$
	Qwen3-8B	Ours	86.23	687	3	6	1.21

Appendix

Table B.5: Main performance comparison on all models across benchmarks (continued; Part 5/6: OpenBooKQA, MMLU).

Dataset	Model	Method	Acc.	Text Tokens	Comm. Passes	AR Steps	Latency
QuAIL	Qwen2.5-7B	Ours	67.07	2388	3	9	1.74
		Latent-MAS	71.87	2387	12	19	4.04
		Text-MAS	68.88	3066	312	336	11.97
		Single	65.60	599	-	7	0.32
		Improve	-1.81	22.1%	99.0%	97.3%	6.9 $\times$
	Qwen3-8B	Ours	77.44	2378	3	6	1.49

Appendix

Table B.6: Main performance comparison on all models across benchmarks (continued; Part 6/6: QuAIL).

Figure B.2: Cumulative token and latency scaling of our method versus autoregressive latent MAS on Qwen3-8B. The left panels show cumulative token consumption and latency, and the right panels show the corresponding differences, computed as ours minus latent MAS.

Multi-Agent Communication

Multi-agent communication studies how autonomous agents exchange information to coordinate decisions, share knowledge, and act together under constraints.

See all 372 papers

LatCom: Cross-Agent Latent Compression for Efficient Multi-Agent Collaboration
Sep 29, 2026Shinan Zhang, Tao Zhang, Qihui Zhu +7Multi-Agent LLM Systems Multi-Agent Collaboration
University of Science and Technology of China · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center
Beyond tokens: a unified framework for latent communication in LLM-based multi-agent systems
Jun 4, 2026Yingzhuo LiuMulti-Agent LLM Systems Multi-Agent Communication
Enabling Agents to Communicate Entirely in Latent Space
Nov 12, 2025Zhuoyun Du, Runze Wang, Huiyu Bai +6Multi-Agent Communication Multi-Agent LLMs
State Key Lab of CAD&CG, Zhejiang University · Zhejiang Key Laboratory of Medical Imaging Artificial Intelligence · Alibaba Group +3

Sep 29, 2026J/K move · Enter open · S save

LatCom: Cross-Agent Latent Compression for Efficient Multi-Agent Collaboration

Shinan Zhang, Tao Zhang, Qihui Zhu, Mengjie Zhang+6

University of Science and Technology of China · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center

Multi-Agent LLM Systems Multi-Agent Collaboration LLM Inference Acceleration Multi-Agent Communication

Open paper AskPDF arXiv

LLM-based multi-agent systems (MAS) increasingly use latent collaboration to avoid the information loss and repeated encoding-decoding overhead of natural-language communication. However, directly forwarding all sender latents makes the receiver-side context scale with both the number of agents and the reasoning length, increasing computation, memory usage, and collaboration latency. A natural solution is latent compression. But we find that cross-agent redundancy remains unresolved in existing latent compression approaches, which typically compress each sender independently and then concatenate the results. We propose LatCom, a cross-agent latent compression framework for efficient multi-agent latent collaboration. LatCom maps multiple sender latents into a fixed number of receiver-readable and task-relevant slots. Rather than reconstructing all sender hidden states, it optimizes the compressed latents for receiver-side task utility. LatCom trains the compressor in two stages: single-sender readability learning first establishes a latent interface interpretable by the frozen receiver, and multi-sender fusion learning then trains the compressor to fuse complementary evidence and remove redundancy across agents. Experiments on multiple benchmarks with Qwen3-4B show that LatCom achieves an average 2.46x inference speed-up over LatentMAS and reduces output token usage by 70.3% while maintaining comparable average accuracy.

Abstract

Multi-Agent Communication

Explore similar work

LatCom: Cross-Agent Latent Compression for Efficient Multi-Agent Collaboration

Beyond tokens: a unified framework for latent communication in LLM-based multi-agent systems

Enabling Agents to Communicate Entirely in Latent Space