The evolution of Large Language Models (LLMs) from passive text processors to autonomous agents has established planning as a core component of modern intelligence. However, achieving generalized planning remains elusive, not only by the scarcity of high-quality interaction data but also by inherent conflicts across heterogeneous planning tasks. These challenges result in models that excel at isolated tasks yet struggle to generalize, while existing multi-task training attempts suffer from gradient interference. In this paper, we present MagicAgent, a series of foundation models specifically designed for generalized agent planning. We introduce a lightweight and scalable synthetic data framework that generates high-quality trajectories across diverse planning tasks, including hierarchical task decomposition, tool-augmented planning, multi-constraint scheduling, procedural logic orchestration, and long-horizon tool execution. To mitigate training conflicts, we propose a two-stage training paradigm comprising supervised fine-tuning followed by multi-objective reinforcement learning over both static datasets and dynamic environments. Empirical results show that MagicAgent-32B and MagicAgent-30B-A3B achieve superior performance across diverse open-source benchmarks (e.g., 75.1% on Worfbench and 86.9% on BFCL-v3), as well as strong results on our in-house MagicEval benchmarks, substantially outperforming existing sub-100B models and surpassing leading ultra-scale models, including GPT-5.2, Kimi-K2 and GLM-4.7.
Figures & tables
Figure 1: Benchmark performance of MagicAgent.
Figure 2: The synthesis pipeline of hierarchical task decomposition.
Figure 3: The synthesis pipeline of tool use planning.
Figure 4: The synthesis pipeline of multi-constrain scheduling.
Figure 5: The synthesis pipeline of procedural logic orchestration.
Figure 6: The synthesis pipeline of long-horizon tool execution.
Figure 7: Overview of the MagicAgent training framework.
Table 2: Benchmark Performance Comparison Across Various Models. Bold indicates the best performance in each benchmark, while Underline indicates the second-best performance.
Models
General
Dependency
Condition
Context Inheritance
Step
Embedding
LLM
Step
Embedding
LLM
Step
Embedding
LLM
Step
Embedding
LLM
Ultra-Scale Models
GPT-5.2
36.8
89.7
92.0
28.5
89.9
96.5
51.0
89.9
85.3
33.5
89.7
91.8
Kimi-K2-Instruct
85.7
91.9
86.7
79.5
91.7
88.1
87.0
91.0
80.2
88.7
91.7
88.3
GLM-4.7
87.9
97.0
88.1
82.5
96.8
89.1
85.5
96.6
83.2
94.6
96.6
88.2
DeepSeek-V3.1-nothink
81.0
96.8
84.0
64.0
96.9
88.3
65.0
96.5
77.0
89.5
96.3
89.9
Table 3: MagicEval-Plan Performance Comparison Across Various Models. Bold indicates the best performance in each benchmark, while Underline indicates the second-best performance.
Models
General
Dependency
Condition
Context Inheritance
Name
Argument
Name
Argument
Name
Argument
Name
Argument
Ultra-Scale Models
GPT-5.2
74.6
55.8
67.8
46.5
73.5
58.2
71.0
55.5
Kimi-K2-Instruct
89.9
66.8
91.0
58.3
88.7
71.0
90.3
67.5
GLM-4.7
89.1
70.6
85.3
56.5
82.6
65.6
85.5
66.0
DeepSeek-V3.1-nothink
86.8
65.0
86.8
54.8
86.6
66.5
86.3
62.0
Table 4: MagicEval-Tool Performance Comparison Across Various Models. Bold indicates the best performance in each benchmark, while Underline indicates the second-best performance.
Models
Methods
IID Succ.∗
IID Succ.
OOD Succ.∗
OOD Succ.
Ultra-Scale Models
GPT-4o
ReAct
57.3
-
66.0
-
GPT-5.2
ReAct
63.5
-
62.6
-
DeepSeek-V3.1
ReAct
67.1
-
76.1
-
DeepSeek-R1
ReAct
68.8
-
70.2
-
Qwen3-235B-A22B-Instruct
ReAct
57.1
-
55.9
-
Table 5: Performance comparison in the online ALFWorld environment. We report two success metrics: Succ.∗ and Succ. . The table is organized into two parts: (i) a ReAct-based comparison across ultra-scale, large-scale, and MagicAgent series, and (ii) a various training-algorithm evaluation on the Qwen2.5 backbone. Bold and underline indicate the best and second-best results, respectively, applied separately within these two parts. Results for baseline methods (ReAct, Agentgym, SFT, GRPO, GiGPO, RLVMR, EPO) are taken from the EPO paper. The gray color indicates that GIGPO and RLVMR employ the Qwen2.5-7B model, whereas EPO and our proposed χ PO use the smaller-parameter Qwen2.5-3B model while achieving comparable performance. We run our own implementations of χ PO across multiple random seeds to ensure stable results.
Figure 8: RL training dynamics analysis. (a) The entropy at the think level is approximately an order of magnitude higher than that at the action level. (b) Both online and offline RL exhibit steadily increasing reward accumulation over the course of training.
Method
WorfBench
NaturalPlan
τ2 -Bench
F1 Chain
F1 Graph
Trip
Meeting
Calendar
Retail
Airline
BP
79.0
68.4
39.5
59.5
54.5
20.8
39.5
BP + LBLmicro
78.5
69.0
38.1
57.8
52.4
43.9
47.0
BP + VarianceLoss
78.5
68.5
39.8
57.4
53.9
28.9
40.0
BP + LBLseq
77.9
68.4
38.1
55.5
54.5
26.1
41.0
BP + LBLgbl
79.3
70.2
40.0
58.5
53.2
45.0
44.0
Table 6: Ablation Study of Joint Optimization Strategy for MoE Supervised Fine-Tuning. Bold indicates the best performance in each benchmark, while Underline indicates the second-best performance.
Figure 9: MaxVio value under various load-balanced strategies. (a) illustrates the MaxVio curves for LBLmicro , LBLgbl , LBLseq and joint optimization strategy. (b) presents the MaxVio metrics for the baseline, z-loss, Expert Capacity, and Variance Loss methods.
Figure 10: Expert frequency heatmaps under different agent tasks on NaturalPlan. (a) Calendar task, (b) Meeting task, and (c) Trip task. The distinct expert utilization patterns across tasks indicate the emergence of stable task-specific expert specialization during training.
Model
Avg TTFT (s)
Avg TPOT (s)
Avg Latency (s)
Concurrency
MagicAgent-32B
0.498
0.016
16.722
1
MagicAgent-30B-A3B
0.341(-31.5%)
0.007(-56.3%)
7.582(-54.7%)
1
MagicAgent-32B
0.785
0.018
19.586
10
MagicAgent-30B-A3B
0.558(-28.9%)
0.011(-38.9%)
11.783(-39.9%)
10
MagicAgent-32B
2.851
0.024
27.855
50
MagicAgent-30B-A3B
1.153(-59.6%)
0.016(-33.3%)
17.891(-35.8%)
50
Table 7: Inference Efficiency Comparison between Dense and MoE Models across Various Concurrency Levels.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Parameter
Default Value
Description
max_length
32768
Maximum prompt length and completion length
num_train_epochs
1
Number of training epochs
weight_decay
0.01
Weight decay coefficient
adam_beta1
0.9
Adam optimizer beta1 parameter
adam_beta2
0.95
Adam optimizer beta2 parameter
lr_scheduler_type
cosine
Learning rate scheduler type
Appendix
Table 8: Training parameters for the MagicAgent-32B and MagicAgent-30B-A3B models.
Figure 11: A demo case of Tool-calling Agent.
Figure 12: A demo case of Travel Planning Agent.
Figure 13: A demo case of Intent-Routing Agent in Multi-Agent architectures.
Planning is central to LLM agents: before acting, an agent must decompose goals, select tools, reason over constraints, and decide when a task is infeasible. Yet existing agent evaluations often report only end-to-end success, making it difficult to determine whether failures stem from planning or execution. We introduce Agent Planning Benchmark (APB), a planning-specific diagnostic benchmark with 4,209 multimodal cases across 22 domains and five settings, covering holistic planning, feedback-conditioned step-wise planning, and robustness under extraneous tools, broken tools, and unsolvable tasks. Across 12 MLLMs, APB reveals systematic weaknesses in long-horizon planning, tool-noise robustness, calibrated refusal, and inference-time refinement. We further validate APB on 200 ToolSandbox tasks and 200 τ2-bench tasks, where APB-guided refinement consistently improves plan correctness, plan grade, and downstream execution metrics across three representative models. APB thus serves as an upstream diagnostic complement to execution benchmarks. The APB benchmark and code are available in this URL.
Haoyu Sun, Wenxuan Wang, Mingyang Song +5
Tongji University · Shanghai AI Laboratory · Harbin Institute of Technology +5
Large Language Models (LLMs) typically exhibit a performance profile where reliability degrades as task complexity increases. We address the challenge of generating high-quality natural language executable plans for complex tasks by introducing GRASP, a strategy-aware, multi-stage planning framework. GRASP decouples the planning pipeline across specialized, context-isolated modules: it pre-compiles global macro-guidelines (GenPlan), explores alternative localized strategies within isolated context windows (RevPlan), and independently evaluates trajectories using a multi-criteria discriminator (VerPlan). Empirical evaluations show that GRASP consistently establishes a new state-of-the-art frontier across diverse datasets, yielding substantial accuracy gains over direct LLM planners on Natural Plan Calendar Scheduling (∼12.4\%$$\uparrow), ZebraLogic (∼30.8\%$$\uparrow), and SciBench Math. Crucially, under multi-task scaling-where standard planners suffer immediate performance collapse-GRASP completely flattens the multi-task degradation penalty. In interleaved dual-task environments, GRASP achieves an absolute accuracy gain of up to 16.7% over direct LLM planners. Furthermore, by isolating context and enforcing strict macro-regularization, GRASP outperforms frontier reasoning models (such as GPT-5-mini) by a margin of 14.5%.
Arunabh Srivastava, Mohammad A., Khojastepour +2
Amir · NEC Laboratories America, Inc. · University of Maryland, College Park, MD
Large Language Model (LLM) agents have shown promise in multi-step planning tasks, but existing approaches like LATS (Language Agent Tree Search) and ReAct rely heavily on LLM inference during planning, leading to high computational costs and stochastic behavior. We present GATS (Graph-Augmented Tree Search), a planning framework that combines systematic UCB1-based tree search with a layered world model to eliminate LLM calls during inference while achieving superior planning performance. Our three-layer world model integrates: (L1) exact symbolic action matching, (L2) statistics learned from execution logs, and (L3) LLM-based prediction for unknown actions. On synthetic planning tasks with branching paths and dead-ends, GATS achieves 100% success rate compared to 92 % for LATS and 64% for ReAct. On a comprehensive stress test spanning 12 challenging scenarios -- including coding workflows, web navigation, and long-horizon tasks -- GATS maintains 100% success while LATS drops to 88.9 % and ReAct to 23.9%. GATS requires zero LLM calls per task during planning (vs. 37 per task for LATS) and produces deterministic plans with zero variance across runs. Our results demonstrate that systematic search with learned world models can substantially outperform LLM-guided exploration for agent planning.