In a multi-agent system (MAS), removing an expensive agent call can save tokens without making the answer arrive sooner: latency depends on the longest chain of dependent calls. We present Latency-Aware Multi-agent System (LAMaS), a modular platform for learning low-latency orchestration. A framework can send a query to LAMaS and receive an answer and execution trace, or keep its own graph builder and use the platform's latency-optimized orchestration layer. During training, the layer gives operators on the critical path the greatest latency penalty while targeting a floor on expected accuracy. During inference, a lightweight controller uses partial results to decide whether to skip work that has yet to start. Across four benchmarks, LAMaS reduces end-to-end latency by over 50% relative to the evaluated learning-based MAS baselines while maintaining competitive or better accuracy. GDesigner and AgentDropout retain their graph builders and train the controller on their own traces; both achieve lower latency across the same benchmarks.
Figures & tables
Figure 1: LAMaS connects agent frameworks to model servers and returns an answer and execution trace for each query. We integrate the full pipeline into GDesigner and AgentDropout, retaining each framework’s graph builder and retraining the execution controller on its traces.
Figure 2: Two execution graphs that produce the same answer at similar cost but differ in latency. Nodes denote agent invocations and arrows denote data flow; red edges mark the critical execution path, the longest chain of dependencies that must complete before an answer is available. Graph B has a shorter critical path and therefore lower wall-clock latency than Graph A.
Figure 3: Training and inference in the LAMaS orchestration layer. (1) Material: A library of reusable operators and tools. (2) Training: Phase 1 performs latency-aware graph optimization via Lagrangian-constrained learning and critical-path credit assignment, where S0 denotes the target accuracy floor enforced during optimization. Phase 2 trains a lightweight execution controller on the orchestrator’s traces to predict the control advantage. (3) Inference: The trained orchestrator first constructs the execution graph; the execution controller then performs online execution control, monitoring operator completions and adaptively eliminating redundant future agent interactions.
Dataset
Method
Acc (%) ↑
Cost ( ×10−4 ) ↓
Latency (s) ↓
CP Tokens ↓
GSM8K
Generate
87.46±0.20
2.87±0.05
5.43±0.18
280±8
CoT
88.97±0.43
4.53±0.04
6.92±0.21
331±15
SC
89.86±0.19
18.61±0.09
10.64±0.27
591±22
GDesigner
93.30±0.93
17.86±0.28
32.31±1.07
1624±29
AgentDropout
93.11±0.27
53.81±6.78
81.24±12.74
4098±538
MaAS
93.36±0.72
31.52±0.78
48.11±3.49
2501±53
Table 1: Comparison with non-MAS and MAS baselines across four benchmarks. Each cell reports mean ± standard deviation across 3 independent runs. For MAS methods, the best mean in each column per dataset is shown in bold . Cost is reported in units of ×10−4 USD. Latency is the average end-to-end wall-clock time per query (in seconds). CP Tokens denotes the average number of output tokens on the critical execution path.
Dataset
Method
Acc (%) ↑
Cost ( ×10−4 ) ↓
Latency (s) ↓
GSM8K
LAMaS
93.65
7.71
11.73
w/o controller
93.27
9.43
13.84
w/o CP Credit
92.73
8.71
12.93
w/o latency obj. ( λt=0 )
93.59
9.71
15.83
naive lat. only
91.85
9.86
15.52
HumanEval
LAMaS
95.42
12.92
21.00
Table 2: Ablation study of inference-time execution control, critical-path (CP) credit assignment, and the latency objective across four benchmarks. Cost is in units of ×10−4 USD; latency is end-to-end wall-clock time per query (in seconds).
Dataset
Method
Acc (%) ↑
Cost ( ×10−4 ) ↓
Latency (s) ↓
CP Tokens ↓
GSM8K
GDesigner
93.30
17.86
32.31
1624
GDesigner+LAMaS
93.27
12.08
20.55
1138
AgentDropout
93.11
53.81
81.24
4098
AgentDropout+LAMaS
93.18
39.67
58.14
2989
HumanEval
GDesigner
85.50
19.86
49.68
2983
GDesigner+LAMaS
85.75
12.93
30.85
1571
Table 3: Transferability of the full LAMaS pipeline (training-time graph optimization, critical-path credit assignment, and inference-time execution control) to other learning-based MAS architectures. The controller is retrained on each architecture’s own traces (Phase 2 of the LAMaS training procedure). Latency is the average end-to-end wall-clock time per query (in seconds). CP Tokens denotes the average critical-path output tokens.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Domain
Dataset
#Train
#Test
Metric
Code Generation
HumanEval
33
131
pass@1
Math Reasoning
GSM8K
264
1055
Accuracy
MATH
119
486
Accuracy
Knowledge Reasoning
MMLU-Pro
125
500
Accuracy
Appendix
Table 4: Dataset statistics.
Setting
Pearson r
Spearman ρ
MAS methods × 4 benchmarks
0.995
0.985
Appendix
Table 5: Correlation between CP Tokens and measured wall-clock latency across MAS-method/benchmark cells in Table 1 .
Benchmark
Decision Acc (%) ↑
Precision ↑
Recall ↑
F1 ↑
HumanEval
73.5
0.72
0.66
0.69
MMLU-Pro
70.8
0.69
0.63
0.66
Appendix
Table 6: Decision-level agreement between the learned execution controller and the oracle stopping rule on HumanEval and MMLU-Pro. The positive class is the “stop” decision.
Benchmark
Controller
Acc (%) ↑
Latency (s) ↓
HumanEval
No controller
95.17
24.47
Learned
95.42
21.00
Oracle
95.67
18.84
MMLU-Pro
No controller
66.00
17.94
Learned
66.27
14.81
Oracle
66.73
13.07
Appendix
Table 7: End-to-end accuracy and latency under different controllers on HumanEval and MMLU-Pro. The “Learned” rows correspond to the LAMaS configuration in Table 2 , and “No controller” rows correspond to the w/o controller ablation.
Figure 4: Average operator selection probabilities on HumanEval after inserting an unseen operator, Debate , at inference time. The unseen operator receives non-trivial probability mass comparable to several seen operators, indicating that the learned policy can still reasonably select operators not exposed during training.
Dataset
λt
Acc (%) ↑
Latency (s) ↓
HumanEval
0
95.67
35.62
0.001
95.93
23.50
0.005 ⧫
95.42
21.00
0.01
94.91
13.91
0.05
93.89
11.29
MMLU-Pro
0
66.33
22.46
Appendix
Table 8: Sensitivity to the latency penalty weight λt . Increasing λt consistently reduces latency, with mild accuracy variation across the sweep.
Dataset
τ
Acc (%) ↑
Latency (s) ↓
HumanEval
0.1
93.89
13.57
0.2
94.91
15.35
0.3 ⧫
95.42
21.00
0.4
95.67
22.80
MMLU-Pro
0.1
65.13
13.42
0.2
65.93
14.23
Appendix
Table 9: Sensitivity to the activation threshold τ . A higher threshold activates more operators per step, increasing accuracy and latency. A lower threshold produces leaner graphs with lower latency but reduced accuracy.
Dataset
L
Acc (%) ↑
Latency (s) ↓
HumanEval
2
94.15
12.85
4 ⧫
95.42
21.00
6
95.67
23.50
8
95.93
25.54
MMLU-Pro
2
64.93
12.81
4 ⧫
66.27
14.81
Appendix
Table 10: Sensitivity to the number of supernet layers L . More layers provide greater reasoning depth and slightly higher accuracy, but increase latency due to deeper critical paths.
Dataset
S0/Sref
Acc (%) ↑
Latency (s) ↓
HumanEval
0.95
94.66
17.85
1.00 ⧫
95.42
21.00
1.02
95.67
22.00
1.05
96.18
23.54
MMLU-Pro
0.95
65.40
12.60
1.00 ⧫
66.27
14.81
Appendix
Table 11: Sensitivity to the accuracy floor S0 . We vary S0 as a fraction of the architecture’s reference accuracy Sref ( S0=Sref is the default, marked by ⧫ ).
Dataset
Backbone
Method
Acc (%) ↑
Latency (s) ↓
HumanEval
qwen3.5-flash
MaAS
94.91
100.87
LAMaS
96.95
38.93
qwen3.5-9b
MaAS
95.17
105.18
LAMaS
95.93
42.18
MMLU-Pro
qwen3.5-flash
MaAS
82.40
174.60
LAMaS
82.93
98.04
Appendix
Table 12: Generalizability across Qwen backbones and serving setups: closed-source qwen3.5-flash served through OpenRouter and open-weight qwen3.5-9b served locally with vLLM. Latency is end-to-end wall-clock time per query (s). Best mean among MAS methods per column per dataset is in bold .
Dataset
Method
Acc (%) ↑
Cost ( ×10−4 ) ↓
Latency (s) ↓
GSM8K
Tool-augmented CoT
89.79
11.38
18.27
LAMaS
93.65
7.71
11.73
HumanEval
Test-time repair
86.51
6.35
15.43
LAMaS
95.42
12.92
21.00
MMLU-Pro
Self-Refine
57.13
5.51
15.19
LAMaS
66.27
10.93
14.81
Appendix
Table 13: Comparison with single-agent agentic baselines. LAMaS rows are taken from Table 1 . Cost is in units of ×10−4 USD; latency is end-to-end wall-clock time per query (in seconds).
Dataset
Method
Acc (%) ↑
Cost ( ×10−4 ) ↓
Latency (s) ↓
GSM8K
GDesigner
93.30
17.86
32.31
GDesigner + consensus-2 stop
92.61
13.97
26.92
GDesigner + half-agents
91.37
11.24
21.37
GDesigner+LAMaS
93.27
12.08
20.55
MMLU-Pro
GDesigner
64.27
25.03
46.66
GDesigner + consensus-2 stop
62.93
19.02
39.48
Appendix
Table 14: GDesigner under two naive inference-time latency-cutting rules. The GDesigner and GDesigner+LAMaS rows are taken from Tables 1 and 3 . Cost is in units of ×10−4 USD; latency is end-to-end wall-clock time per query (in seconds).