Latent-DARM: Bridging Discrete Diffusion And Autoregressive Models For Reasoning

Lina Berrayana; Ahmed Heakl; Abdullah Sohail; Thomas Hofmann; Salman Khan; Wei Chen

cs.LGMar 10, 2026

Latent-DARM: Bridging Discrete Diffusion And Autoregressive Models For Reasoning

Authors: Lina Berrayana, Ahmed Heakl, Abdullah Sohail, Thomas Hofmann, Salman Khan, Wei Chen

Organizations: EPFL · MBZUAI · ETH Zürich · Microsoft Research Asia

Abstract

Most multi-agent systems rely exclusively on autoregressive language models (ARMs) that are based on sequential generation. Although effective for fluent text, ARMs limit global reasoning and plan revision. On the other hand, Discrete Diffusion Language Models (DDLMs) enable non-sequential, globally revisable generation and have shown strong planning capabilities, but their limited text fluency hinders direct collaboration with ARMs. We introduce Latent-DARM, a latent-space communication framework bridging DDLM (planners) and ARM (executors), maximizing collaborative benefits. Across mathematical, scientific, and commonsense reasoning benchmarks, Latent-DARM outperforms text-based interfaces on average, improving accuracy from 27.0% to 36.0% on DART-5 and from 0.0% to 14.0% on AIME2024. Latent-DARM approaches the results of state-of-the-art reasoning models while using less than 2.2% of its token budget. This work advances multi-agent collaboration among agents with heterogeneous models.

Figures & tables

Figure 1: Accuracy–cost trade-offs across planner–executor configurations. Here DDLM refers to a LLada-8B-Instruct and ARM to a Llama-3.2-3B-Instruct model. DDLM

\to

ARM, particularly with latent-space exchange, achieves higher reasoning accuracy at lower token budgets compared to ARM-only models.

Figure 2: Overview of the latent-space collaboration pipeline. A discrete diffusion language model (DDLM) generates a latent plan. The plan is projected directly into the autoregressive model (ARM) embedding space through a learned projector (in blue). The ARM then conditions on the plan and the question to produce the final answer.

Accuracy
Model / Setting	ARC-E	ARC-C	DART-1	DART-2	DART-3	DART-4	DART-5	AIME24	MMLU
DeepSeek-R1 (ARM only )	94.0	88.0	89.0	81.5	85.5	75.0	61.5	28.5	60.0
Qwen3-1.7B (ARM only)	92.0	86.0	89.5	86.0	83.0	73.0	49.0	8.5	68.5
Llama-3.2-3B (ARM only)	87.5	79.5	44.5	35.5	28.0	34.5	36.5	3.5	54.0
LLaDA-8B $\to$ Llama-3.2-3B (text-space)	90.5	82.5	53.5	43.0	35.5	30.0	27.0	0.0	52.5
LLaDA-8B $\to$ Llama-3.2-3B (latent-space)

Table 1: Evaluation of DeepSeek-R1-Distill-Qwen-7B and Qwen3-1.7B on reasoning benchmarks, including text-space vs. latent-space collaboration.

Figure 3: Accuracy comparison of text-space vs. latent-space collaboration.

Figure 4: Diagnostic configurations for attributing errors to planner or executor. Setup X tests whether failures stem from the planner: if replacing the diffusion planner (DDLM) with an autoregressive planner (ARM) fixes the output, the error is attributed to the DDLM. Setup Y tests executor reliability: if a diffusion executor succeeds where an ARM executor fails, the limitation lies in the executor.

Figure 5: Planner vs. executor failures in text- vs. latent-space collaboration. Results for LLaDA-8B-Instruct and Llama-3.2-3B-Instruct. Latent-space collaboration substantially reduces planning errors compared to text-space.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Simulation	D-3	R-4	LR-n
	$\uparrow$	$\downarrow$	$\downarrow$
Qwen2.5-7B-Instruct (Baseline)	98.47	3.05	0.64
LLaDA-8B-Instruct (256 tokens)	83.05	10.52	6.74
LLaDA-8B-Instruct (128 tokens)	93.15	5.33	3.43
LLaDA-8B-Instruct (64 tokens)	96.85	3.37	1.33
Dream-v0-7B-Instruct (256 tokens)	62.02	3.26	2.15

Appendix

Table 2: Repetition Evaluation

LLaDA-8B + Llama-3.2-3B (Text-Space Pipeline)
Benchmark	Planning Failures (%)	Execution Failures (%)	Error Gap (%)
DART-1	41.50	43.64	2.14
DART-2	21.43	24.14	2.71
DART-3	13.60	16.92	3.32
DART-4	11.11	15.79	4.68
DART-5	10.12	5.63	4.49

Appendix

Table 3: Performance comparison of planner vs. executor issues for LLaDA-8B-Instruct and Llama-3.2-3B-Instruct under Text-Space vs. Latent-Space collaboration.

ARMs only
Model / Combination	Setup	ARC-E	ARC-C	DART-1	DART-2	DART-3	DART-4	DART-5	AIME	MMLU
Qwen2.5-3B	ARM	91.0	86.5	58.0	34.0	29.5	17.5	11.5	1.0	61.0
	ARM $\to$ ARM	89.5	81.5	64.0	49.0	30.5	24.0	16.0	0.0	59.0
Dream-v0-7B + Qwen2.5-3B	DDLM $\to$ ARM	92.0	86.0	65.5	40.5	31.0	25.0	12.5	0.5	60.5
LLaDA-8B + Qwen2.5-3B	DDLM $\to$ ARM	88.5	85.0	68.0	44.5	29.0	23.0	14.0	0.5	55.5
Qwen2.5-7B	ARM	94.5	90.5	68.0	48.5	35.0	35.0	19.5	2.5	67.0

Appendix

Table 4: Consolidated evaluation results on text-space collaboration across all model combinations and reasoning benchmarks.

\dagger

Evaluated with enable_thinking=True ,

\ddagger

with enable_thinking=False .

Multi-Agent Reasoning

Multi-Agent Reasoning studies how multiple AI agents share evidence, model one another, and coordinate decisions on complex tasks.

See all 229 papers

Beyond the Sequentiality Bottleneck: A Hybrid Diffusion-Autoregressive Framework for Efficient Multimodal Reasoning
Oct 20, 2026Wei Cai, Jian Zhao, Yuchen Yuan +1Hybrid Language Models Parallel Reasoning in Language Models
Peking University · Institute of Artificial Intelligence (TeleAI), China Telecom
DiffuPlan: Editable Planning State for Diffusion Language Model Agents
Oct 17, 2026Qisheng Zhou, Zhen Xiong, Qiaoyu TanLLM Agents LLM Agent Planning
New York University Shanghai · New York University
Think First, Diffuse Fast: Improving Diffusion Language Model Reasoning via Autoregressive Plan Conditioning
Feb 20, 2026Earl J St SauverDiffusion Language Models LLM Planning
Independent Researcher

Oct 20, 2026J/K move · Enter open · S save

Beyond the Sequentiality Bottleneck: A Hybrid Diffusion-Autoregressive Framework for Efficient Multimodal Reasoning

Wei Cai, Jian Zhao, Yuchen Yuan, Xuelong Li

Peking University · Institute of Artificial Intelligence (TeleAI), China Telecom

Hybrid Language Models Parallel Reasoning in Language Models Reasoning for VLMs Diffusion Language Models RL for Diffusion Models

Open paper AskPDF arXiv

Autoregressive models (ARMs) face a "Sequentiality Bottleneck" in reasoning, while diffusion language models (DLMs) offer parallelism but lack semantic coherence. To addresses both limitations, we propose Collaborative Reasoning Asymmetric Learning (CoRAL), a hybrid framework via an Asymmetric Planner-Executor (APE) architecture: a DLM Planner generates K diverse reasoning trajectories in parallel, while a large ARM Executor selects the optimal plan and synthesizes the final answer. To train the Planner effectively, we introduce Reward-Guided Denoising for CoT (RDC), which integrates outcome-based rewards into the diffusion process to steer it toward goal-directed reasoning. Experiments on vision-language benchmarks show that CoRAL significantly outperforms prior diffusion-based methods and achieves competitive performance against strong ARMs, achieving both superior efficiency and quality.

Abstract

Multi-Agent Reasoning

Explore similar work

Beyond the Sequentiality Bottleneck: A Hybrid Diffusion-Autoregressive Framework for Efficient Multimodal Reasoning

DiffuPlan: Editable Planning State for Diffusion Language Model Agents

Think First, Diffuse Fast: Improving Diffusion Language Model Reasoning via Autoregressive Plan Conditioning