cs.AISep 30, 2026

STRATA: Self-Learning Through Role-Aligned Tiered Agents for Real-Time Strategy Games

Authors: Xinhe Tian, Xiaoyue Zhang, Ziyou Zhang, Jiacheng Li, Xiaoqiang Jin, Qianchuan Zhao, Gaochen Cui

Organizations: School of Instrumentation Science and Optoelectronic Engineering, Beijing University of Aeronautics and Astronautics · Tsinghua University · University of Chinese Academy of Sciences · Department of Automation, Tsinghua University · CFINS, Tsinghua University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Real-time strategy (RTS) games require agents to coordinate economic development, production and construction, base defense, unit organization, and attack timing over long matches. Existing studies have applied large language models to command decision-making in RTS games, enabling agents to read textual game states and generate high-level plans. However, long inference latency can cause them to miss critical tactical events. The complexity and tactical diversity of full RTS matches also leave existing systems heavily dependent on manually written experience-based prompts, with limited ability to learn continuously from past games. We present STRATA, a role-aligned hierarchical system with cross-game self-learning for Red Alert. STRATA assigns in-game strategic, logistical, and tactical decisions to a Strategic Agent (SA), Logistics Agent (LA), and Tactical Agent (TA), respectively. The SA generates high-level directives based on the global game state and relevant experience cards, while the LA and TA handle logistics and tactical execution. After each match, a Review Agent (RA) derives candidate experience from game traces, validates and revises it using evidence from subsequent matches, and compresses strategic experience supported across multiple games into concise experience cards for SA retrieval. We evaluate STRATA through the formation of experience cards, full-match comparisons before and after learning, and experience learning against AI opponents with different play styles. Under a fixed scenario, using the learned experience cards increases the observed win rate from 30% to 100%. Sequential learning against AI opponents with different play styles also produces distinct long-term strategic experience.

Figures & tables

Figure 1: Overview of STRATA. The system combines role-aligned hierarchical decision-making during each match with a closed loop for post-game experience formation, cross-game validation, and retrieval in later matches.
Experience themeRush conditionTurtle condition
Vehicle productionLower the production threshold under a thin economy. With fewer than four harvesters, allow cash to fall to about 100 and queue one vehicle at a time to prevent the war factory from idling.Use stage-dependent cash floors. Keep about 200 before the second refinery. Afterward, produce vehicles while cash exceeds 1,500 and stop near 800.
Economic and production expansionPrioritize the second refinery over additional production capacity, radar, and technology to stabilize the basic economy before further development.Under low pressure, prioritize a third refinery and another war factory so that economy and production capacity expand together.
Post-technology attackProduce advanced units in short batches and wait for an attack window.Reduce prolonged rallying and waiting. When at least six combat units are available and few opponents are visible, the SA is more likely to issue a proactive attack directive.
Table 3: Main revisions produced from the same initial experience cards under Rush and Turtle conditions.

Explore similar work

CardsList
Jun 17, 2026cs.AI

RTSGameBench: An RTS Benchmark for Strategic Reasoning by Vision-Language Models

Modern Vision-Language Models (VLMs) often struggle with strategic reasoning, i.e., anticipating and influencing other agents' actions, under uncertainty in competitive and cooperative settings. Real-time strategy (RTS) games can be a natural testbed for diagnosing this limitation, as they demand coordination with allies, adaptation to opponents' strategy, and long-horizon planning under partial observability. However, existing RTS benchmarks offer limited evaluation scope, lack systematic competency diagnosis, and remain fixed in the pre-designed scenario coverage. To address these limitations, we present RTSGameBench, which is built on Beyond All Reason, a large-scale RTS game with an expanded battlefield that demands broader strategy diversity than the existing testbeds. The proposed benchmark provides evaluations through diverse gameplay across various matchup structures, diagnostic assessment via mini-games, each targeting an individual strategic competency, and extensible coverage via a self-evolving generation framework that converts free-form queries into new mini-games, improving over successive cycles. Additionally, for VLMs to operate in large-scale RTS games, we provide RTSGameAgent that manages units by an FSM with agentic memory. We empirically validate that multiple state-of-the-art VLMs do not perform well when matchups demand tighter coordination, multiagent coordination and when task scale increases.

San Kim, Daechul Ahn, Reokyoung Kim +3

Seoul National University

Strategic ReasoningAgentic BenchmarksReasoning BenchmarkMulti-Agent CoordinationAgentic Memory
AskPDF
Sep 28, 2026cs.AI

SAGE: Structured Strategic Reasoning for Efficient LLM Game Playing

A strong LLM strategic agent should reason prospectively over uncertain futures, adapt its strategy to opponents' behavioral tendencies, and continuously recalibrate its decision process from interaction experience. However, incorporating these sources in free-form reasoning could lead to unsupported strategic assumptions, inconsistent opponent estimates, and harmful interference from irrelevant historical interactions. To address these issues, we propose SAGE, a training-free inference-time framework that structures LLM strategic reasoning around three coordinated operations: anchor, adapt, and recalibrate. SAGE first anchors reasoning to an equilibrium policy that provides a strategically valid prior. It then conditions deviations from this anchor on a soft belief over opponent behavioral tendencies, enabling opponent-specific exploitation. Finally, SAGE distills strategically related interactions into counterfactual hypotheses about previously missing considerations, allowing past experience to recalibrate the model's reasoning. We evaluate SAGE on three repeated imperfect-information games: Leduc Hold'em, Liar's Dice, and Goofspiel, against various opponent types in each game. Compared with reasoning-intensive LLM agents, including Suspicion-Agent, ReTA, Agent-Pro, EMO, and Hypothetical Minds, SAGE achieves up to a 127.6% payoff improvement in Liar's Dice while reducing input and output token usage by up to 80% and 90%, respectively. In direct match-up play, it attains non-negative mean payoff against 5/10, 8/10, and 8/10 evaluated opponents in Leduc Hold'em, Liar's Dice, and Goofspiel, respectively, while using relatively fewer tokens. Code is available at https://github.com/chenzhwsysu57/SAGE.

Zhiwei Chen, Tianchun Wang, Zhongtao Rao +3

The Hong Kong University of Science and Technology (Guangzhou) · Johns Hopkins University · Microsoft +2

Strategic ReasoningLarge Language Model AgentsExploitation
AskPDF
May 7, 2026cs.CL

StraTA: Incentivizing Agentic Reinforcement Learning with Strategic Trajectory Abstraction

Large language models (LLMs) are increasingly used as interactive agents, but optimizing them for long-horizon decision making remains difficult because current methods are largely purely reactive, which weakens both exploration and credit assignment over extended trajectories. In this work, we present Strategic Trajectory Abstraction (StraTA), a simple framework that introduces an explicit trajectory-level strategy into agentic reinforcement learning (RL). StraTA samples a compact strategy from the initial task state, conditions subsequent actions on that strategy, and trains strategy generation and action execution jointly with a hierarchical GRPO-style rollout design, further enhanced by diverse strategy rollout and critical self-judgment. Experiments on ALFWorld, WebShop, and SciWorld show that StraTA consistently improves both sample efficiency and final performance over strong baselines. StraTA reaches success rates of 93.1% on ALFWorld and 84.2% on WebShop. On SciWorld, StraTA attains a 63.5% overall score, outperforming frontier closed-source models.

Xiangyuan Xue, Yifan Zhou, Zidong Wang +5

1The Chinese University of Hong Kong · 2Shanghai Artificial Intelligence Laboratory · University of Georgia +2

Agentic Reinforcement LearningAgentic Learning
AskPDF