ProMAS: Proactive Error Forecasting for Multi-Agent Systems Using Markov Transition Dynamics

Xinkui Zhao; Sai Liu; Yifan Zhang; Qingyu Ma; Guanjie Cheng; Naibo Wang; Chang Liu

cs.AIMar 12, 2026

ProMAS: Proactive Error Forecasting for Multi-Agent Systems Using Markov Transition Dynamics

Authors: Xinkui Zhao, Sai Liu, Yifan Zhang, Qingyu Ma, Guanjie Cheng, Naibo Wang, Chang Liu

Organizations: Zhejiang University.

Abstract

The integration of Large Language Models into Multi-Agent Systems (MAS) has enabled the so-lution of complex, long-horizon tasks through collaborative reasoning. However, this collec-tive intelligence is inherently fragile, as a single logical fallacy can rapidly propagate and lead to system-wide failure. Most current research re-lies on post-hoc failure analysis, thereby hinder-ing real-time intervention. To address this, we propose PROMAS, a proactive framework utiliz-ing Markov transitions for predictive error anal-ysis. PROMAS extracts Causal Delta Features to capture semantic displacement, mapping them to a quantized Vector Markov Space to model reasoning as probabilistic transitions. By inte-grating a Proactive Prediction Head with Jump Detection, the method localizes errors via risk acceleration rather than static thresholds. On the Who&When benchmark, PROMAS achieves 22.97% step-level accuracy while processing only 27% of reasoning logs. This performance rivals reactive monitors like MASC while reducing data overhead by 73%. Although this strategy entails an accuracy trade-off compared to post-hoc meth-ods, it significantly improves intervention latency, balancing diagnostic precision with the real-time demands of autonomous reasoning.

Figures & tables

Figure 1: Overview of the ProMAS . The architecture is divided into two coupled components: (Left) Proactive Inference: The dialogue history $H_{t-1}$ is encoded into a latent state $s_{t-1}$ to predict the cluster distribution and transition risk $R_{t}$ of the next action. A Jump Detection mechanism identifies logic breaches by monitoring risk mutations. (Right) Multi-stage Training: (i) Causal Manifold Learning aligns semantic displacement features $\Delta_{t}=s_{t}-s_{t-1}$ via triplet loss; (ii) Transition Modeling constructs discrete Markov matrices from successful and failed trajectories; (iii) Proactive Head Optimization trains the model to anticipate future action clusters.

	Method	Algorithm-Generated		Hand-Crafted		Total
	Method	Agent-level	Step-level	Agent-level	Step-level	Agent-level	Step-level
-	Random	29.10	19.06	12.00	4.16	23.71	14.36
Offline	LLM-Based
	All-at-Once	51.12	13.52	53.44	3.51	51.85	10.37
	Step-by-Step	26.02	15.31	53.44	8.77	28.14	13.25
	Binary Search	30.11	16.59	36.21	6.90	32.03	13.54

Table 1: Performance comparison of failure attribution accuracy (%) on the Who&When benchmark. We categorize methods into Post-hoc Analysis (performing diagnosis after task completion) and Proactive Monitoring (identifying breaches during execution).

\mathcal{G}

denotes ground-truth guidance. ProMAS achieves competitive precision while adhering to the strict causality constraint of real-time environments.

Method	Look-ahead	$\eta$ (%) $\downarrow$	Step Acc. $\uparrow$
Random	None	50.0	14.36
ALL-at-Once	Full	100.0	10.37
AgenTracer	Full	100.0	31.89
Famas	Full	100.0	29.35
MASC	None	42.19	21.62
ProMAS	None	26.79	22.97

Table 2: Information Efficiency Analysis. We compare ProMAS with baselines regarding the processed context ratio. Look-ahead indicates if future (post-error) information is required.

\eta

denotes the Information Fraction : the percentage of dialogue processed at the time of detection. ProMAS achieves high accuracy with minimal context.

Split	Backbone	Agent-level	Step-level
Automated	Llama-3.1-8B	46.53	24.75
Automated	Qwen3-8B	42.57	23.76
Handcraft	Llama-3.1-8B	27.66	14.89
Handcraft	Qwen3-8B	36.17	17.02
Total	Llama-3.1-8B	40.54	22.97
Total	Qwen3-8B	40.54	21.62

Table 3: Generalization study of ProMAS .

Figure 2: Ablation study of ProMAS on the Algorithm-Generated split of Who&When benchmark.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Parameter	Value
Model Architecture	Llama 3.1 8B Instruct
Parameter Count	8 Billion
Hidden Dimension ( $D$ )	4096
Precision	BFloat16
Context Window	8192 tokens
Pooling Strategy	Mean of Last 4 Layers

Appendix

Table 4: Hyperparameters of the Frozen LLM Backbone.

	Algorithm-Generated		Hand-Crafted
	GAIA	AssistantBench	GAIA	AssistantBench
Total Number	98	28	30	28
Maximum Agent Number	4	4	5	4
Minimum Agent Number	1	3	1	2
Maximum Log Length	10	10	130	129
Minimum Log Length	5	6	5	8

Appendix

Table 5: Additional details about the Who&When benchmark: We present the total number of tasks for each category, along with the maximum and minimum number of agents and log lengths.

Table 6: Hyperparameter settings for Causal Manifold training and Proactive Estimation.

Figure 3: Impact of the number of clusters ( $K$ ) on Agent-level and Step-level performance. The results indicate that $K=30$ offers the optimal trade-off between semantic resolution and transition robustness.

Language Model Error Detection

Language model error detection identifies unreliable answers, reasoning steps, and actions so systems can flag, verify, or repair them.

See all 90 papers

Towards Self-Improving Error Diagnosis in Multi-Agent Systems
Apr 19, 2026Jiazheng Li, Emine Yilmaz, Bei Chen +1Multi-Agent LLM Systems Multi-Agent Systems
King’s College London · Amazon Alexa AI · University College London
Efficient Failure Management for Multi-Agent Systems with Reasoning Trace Representation
Mar 23, 2026Lingzhe Zhang, Tong Jia, Mingyu Wang +9Agent Failure Analysis Multi-Agent LLM Systems
Peking University Beijing, China · Huawei Technologies Co., Ltd. Beijing, China · Huawei Technologies Co., Ltd. Hong Kong SAR, China +1
Conformal Agent Error Attribution
May 7, 2026Naihe Feng, Yi Sui, Shiyi Hou +2Agent Failure Analysis AI Agent Reliability
Dalhousie University · Halifax, NS, Canada · Layer 6 AI +1

Apr 19, 2026J/K move · Enter open · S save

Towards Self-Improving Error Diagnosis in Multi-Agent Systems

Jiazheng Li, Emine Yilmaz, Bei Chen, Dieu-Thu Le

King’s College London · Amazon Alexa AI · University College London

Multi-Agent LLM Systems Multi-Agent Systems Self-Improving Agents LLM Agent Reliability

Open paper AskPDF arXiv

Large Language Model (LLM)-based Multi-Agent Systems (MAS) enable complex problem-solving but introduce significant debugging challenges, characterized by long interaction traces, inter-agent dependencies, and delayed error manifestation. Existing diagnostic approaches often rely on expensive expert annotation or ''LLM-as-a-judge'' paradigms, which struggle to pinpoint decisive error steps within extended contexts. In this paper, we introduce ErrorProbe, a self-improving framework for semantic failure attribution that identifies responsible agents and the originating error step. The framework operates via a three-stage pipeline: (1) operationalizing the MAS failure taxonomy to detect local anomalies, (2) performing symptom-driven backward tracing to prune irrelevant context, and (3) employing a specialized multi-agent team (Strategist, Investigator, Arbiter) to validate error hypotheses through tool-grounded execution. Crucially, ErrorProbe maintains a verified episodic memory that updates only when error patterns are confirmed by executable evidence, without the need for annotation. Experiments across the TracerTraj and Who&When benchmarks demonstrate that ErrorProbe significantly outperforms baselines, particularly in step-level localization, while the verified memory enables robust cross-domain transfer without retraining.

Abstract

Language Model Error Detection

Explore similar work

Towards Self-Improving Error Diagnosis in Multi-Agent Systems

Efficient Failure Management for Multi-Agent Systems with Reasoning Trace Representation

Conformal Agent Error Attribution