Self-Evolving Multi-Agent Framework for Efficient Decision Making in Real-Time Strategy Scenarios
Authors: Li Ma, Hao Peng, Yiming Wang, Hongbin Luo, Jie Liu, Kongjing Gu, Guanlin Wu, Hui Lin, Lei Ren
Organizations: School of Cyber Science and Technology, Beihang University, Beijing 100191, China · Naval Aviation University, Yantai 264001, China · Military Science Academy, Beijing 100091, China · National University of Defense Technology, Changsha 410073, China · China Academy of Electronics and Information Technology, Beijing 100846, China · School of Automation Science and Electrical Engineering, Beihang University, Beijing 100191, China
Large language models (LLMs) have demonstrated exceptional potential in complex reasoning,pioneering a new paradigm for autonomous agent decision making in dynamic settings. However, in Real-Time Strategy (RTS) scenarios, LLMs suffer from a critical speed-quality trade-off. Specifically expansive state spaces and time limits render inference delays prohibitive, while stochastic planning errors undermine logical consistency. To address these challenges, we present SEMA (Self-Evolving Multi-Agent), a novel framework designed for high-performance, low-latency decision-making in RTS environments. This collaborative multi-agent framework facilitates self-evolution by adaptively calibrating model bias through in-episode assessment and cross-episode analysis. We further incorporate dynamic observation pruning based on structural entropy to model game states topologically. By distilling high dimensional data into core semantic information, this approach significantly reduces inference time. We also develop a hybrid knowledge-memory mechanism that integrates micro-trajectories, macro-experience, and hierarchical domain knowledge, thereby enhancing both strategic adaptability and decision consistency. Experiments across multiple StarCraft II maps demonstrate that SEMA achieves superior win rates while reducing average decision latency by over 50%, validating its efficiency and robustness in complex RTS scenarios.
Figures & tables
Figure 1: The SEMA framework addresses two pivotal challenges for LLMs in RTS environments. First, the massive observational data leads to excessive input sequences, escalating reasoning latency and hindering real-time response. Second, the inherent stochastic bias of LLMs induces inconsistent decision logic, even in the exact same scenario, two distinct decisions can yield diametrically opposed outcomes, such as a shift from victory to defeat. This volatility severely undermines the robustness of agents in complex adversarial settings.
Notation
Definition
Notation
Definition
Notation
Definition
G
Graph structure
o
Local observation
Φ
Spatio-temporal operator
V
Vertex set
N
Multi-agent set
w
Normalized edge weight
E
Edge set
L
Hierarchical storage
Δ
Attribute variation
W
Global weight function
M
Micro-memory block
δ
Variational metric
d
Node degree
E
Macro-experience pool
μ
Pruning threshold
T
Encoding tree
K
External knowledge base
N
Capacity factor
Table 1: List of Notations for SEMA Framework.
Figure 2: Overview of SEMA. First, structural modeling and dynamic pruning are employed to extract core observations, reducing reasoning latency. Second, decision and evaluation agents perform closed-loop calibration via history retrieval to suppress stochastic bias. Finally, the policy agent analyzes episode performance and updates experience, driving the continuous self-evolution of strategic logic.
Method
3m
8m
25m
Flat32
Flat48
Flat64
Simple64
Lv.1
Lv.2
Lv.1
Lv.2
Lv.1
Lv.2
Lv.1
Lv.2
Random
2/16s
2/9s
0/-
0/-
0/-
0/-
0/-
0/-
0/-
0/-
0/-
Rule-Based
50/8s
46/9s
55/14s
58/6m
53/7m
63/8m
58/9m
60/10m
56/11m
61/10m
59/10m
Single-LLM
40/16s
32/12s
34/20s
75/5m
72/7m
74/6m
73/6m
71/7m
70/6m
71/7m
69/9m
TextStarCraft
56/14s
44/14s
38/19s
83/6m
83/6m
87/6m
80/7m
86/7m
82/6m
85/7m
81/8m
HIMA
66/20s
54/19s
10/23s
94/6m
89/6m
100/7m
94/7m
95/6m
89/7m
90/7m
85/8m
Table 2: Win rate(%) and average episode time for successful games across different methods on SMAC and Melee Maps. The time duration listed represents the execution time of a single successful episode; cases where no victory was achieved are denoted by a dash.
Figure 3: Performance comparison of win rate and response time across diverse scenarios. (a) 3m; (b) 8m; (c) 25m; (d) Flat64 (VeryEasy); (e) Flat64 (Easy). For each point, the values in parentheses (x,y) , e.g., (0.49,88) , denote the average response time (s) and the success rate (%) .
Method
3m
8m
Flat64
Simple64
Lv.1
Lv.2
Lv.1
Lv.2
w/o Dynamic Pruning
50 / 1.4s
60 / 1.4s
90 / 1.5s
85 / 1.9s
90 / 1.6s
75 / 2.1s
w/o Evaluation Agent
75 / 0.5s
67 / 0.7s
93 / 0.7s
86 / 0.7s
93 / 0.6s
88 / 0.7s
w/o Policy Agent
47 / 0.5s
52 / 0.6s
83 / 0.8s
78 / 0.7s
82 / 0.8s
72 / 0.7s
SEMA (Ours)
88 / 0.5s
70 / 0.5s
100 / 0.9s
94 / 0.8s
100 / 0.8s
100 / 0.8s
Table 3: Performance comparison of win rate and inference time per step across different components. The win rate serves as a key performance indicator, while the recorded time denotes the duration of a single successful episode.
Figure 4: Ablation study of structural entropy driven pruning on response time across different maps and different levels. (a) 3m; (b) 8m; (c) Flat64(VeryEasy); (d) Flat64(Easy).
Phase
( α,β )
Resource
Building
Unit
Research
Planning
Total
Early
(0.95, 0.05)
36.6
75.9
63.6
76.4
0
19.1
(0, 1)
53.8
59.5
55.8
81.2
0
18.3
Mid
(0.95, 0.05)
84.6
65.6
77.8
100
100
24.0
(0, 1)
29.0
90.7
75.5
99.9
99.7
22.2
Late
(0.95, 0.05)
62.1
71.4
73.0
100
0
19.2
(0, 1)
68.5
84.4
69.3
70.4
0
19.6
Table 4: Analysis of information retention rates across different combat phases and hyperparameter configurations.
Figure 5: Hyperparameter sensitivity analysis and semantic filtering visualization on the Simple64-Lv2 map. (a) and (b) (at frame 30 and frame 200, respectively) visualize the structural entropy-driven dynamic pruning process of observation information, where red regions indicate observation attribute nodes with ΔH below the threshold μ that require pruning, while green regions represent retained attribute nodes; (c) shows the relationship between average win rate and the number of tokens under different capacity factors N .
Method
3m
8m
25m
Flat32
Flat48
Simple64
Lv.1
Lv.2
Lv.1
Lv.2
Lv.1
Lv.2
Single-LLM
7.0k
8.3k
12.1k
14.0k
16.4k
12.1k
12.2k
12.5k
13.3k
TextStarCraft
2.9k
3.4k
5.1k
15.9k
15.4k
15.2k
15.0k
15.2k
15.6k
HIMA
12.1k
13.9k
33.1k
11.1k
11.4k
12.1k
12.8k
10.4k
11.8k
SEMA (Ours)
0.9k
1.5k
2.2k
2.9k
2.9k
2.9k
2.8k
2.9k
2.9k
Table 5: Comparative analysis of token consumption per step across different scenarios.
Figure 6: Snapshots of tactical evolution on the 8m map ( μ=0.48 ). (a) illustrates the disordered state in Episode 2, while (b) demonstrates the collaborative formation and focus fire strategy in Episode 3.
School of Instrumentation Science and Optoelectronic Engineering, Beijing University of Aeronautics and Astronautics · Tsinghua University · University of Chinese Academy of Sciences +2