A deployed LLM coaching agent answers requests whose correct response depends on player state and game version, so advice that is right for one player can be wrong for the next. We describe TactiNavi, the serving system behind the LLM coach of a commercial open-world game, which makes strategy applicability an explicit artifact shared by offline compilation, online execution, failure diagnosis, and release. An offline compiler turns versioned game evidence and expert guidance into strategy packages with explicit prerequisites, the serving plane evaluates those prerequisites as three-valued conditions over structured player state and records decision traces, and a control plane uses the resulting condition-rule dependencies to validate revisions on held-out traffic before release. In a retrospective evaluation of 1,122 logged coaching turns from three deployment periods, condition-bearing packages raise correctness from 57.9% to 82.0% over atomic facts and reduce strategy misapplication under changed player state from 18.1% to 10.4%, while trace-based claim verification reaches 86.4% precision at a median reply latency of 2.07 s. The deployment also showed that most apparent fixes are generation noise, because an unchanged-system rerun recovers 38.4% of failing turns and only one of 15 shipped revision units passes the release checks retrospectively. We distill these results into lessons on treating applicability as a first-class serving object, sizing traces for later diagnosis, and building the null model into the release path.
Figures & tables
Req.
Mechanism
R1
Versioned packages with evidence, prerequisites, and applicable versions (Sec. 4.2 ).
R2
Three-valued conditions over state, evaluated before generation (Sec. 4.3 ).
R3
Traces and condition–rule dependencies for attribution (Sec. 4.4 ).
R4
Scoped validation, counterexamples, and release gates (Sec. 4.5 ).
Table 1: Serving requirements and the mechanisms that address them.
Figure 1: Overview of TactiNavi. The Strategic Knowledge Compiler supplies versioned strategy packages for online condition evaluation. The Agent Execution Plane generates guidance and records decision traces, which support feedback-driven diagnosis. Attributed repair targets enter the Evolution Control Plane for independent validation, and accepted revisions update knowledge and execution components for subsequent coaching.
Knowledge
Corr.
Contr.
Q2
Q3
Pass
Tok.
Viol.
Atomic facts
57.9
0.91
1.02
1.21
41.9
767
10.4
Guide
82.5
0.28
1.27
1.43
62.7
1298
18.1
Package
82.0
0.28
1.32
1.44
65.2
1477
10.4
− cond. rules
75.6
0.43
1.19
1.43
61.4
1246
21.5
Table 2: Knowledge configurations on T3 strategic turns. Corr., Pass, and Viol. are percentages. Viol. is measured on false and unknown variants. Contr. counts contradicted claims and violations. Q2 and Q3 range from 0 to 2. Tok. counts knowledge tokens.
Method
Prec.
Rec.
Rej.
Lag.
Doubt as failure
75.0
100.0
0/12
0.0
Direct feedback
75.6
94.4
1/12
0.0
Category routing
88.0
61.1
9/12
0.0
Who&When
77.8
97.2
2/12
0.0
TactiNavi
86.4
52.8
9/12
26.7
− turn localization
80.8
58.3
7/12
0.0
Table 3: Failure diagnosis on expert-adjudicated T1 doubts. Prec. and Rec. are percentages for founded doubts. Rej. counts rejected unfounded doubts out of 12. Lag. is delayed-feedback localization (%).
Revision
Units
Rec.
Gain
Spec.
Eligible
Unchanged rerun
–
38.4
+0.6
–
–
Deployed units
15
–
–
–
1/15
Planted, entity
3
39.1
−0.6
3/3
0/3
Planted, mechanism
3
40.7
+0.7
0/3
0/3
Table 4: Independent revision assessment on T3 . Rec. is recovery among previously failing turns for the unchanged rerun and among failing exposed turns for planted controls (%). Gain is the paired correctness difference in percentage points, evaluated in scope for planted controls. Spec. counts case-specific revisions, and Eligible counts those passing release checks. Unchanged reruns use S1 , while planted controls use S3 .
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Stage
Unsupported when
Repair owner
Reply rendering
The response misstates or exceeds its reply context
Reply execution
Context assembly
Available relevant evidence is omitted from the reply context
Reply execution
Result merging
A correct result is lost when tool outputs are combined
Upstream execution
Routing
A required tool or knowledge source is not selected
Upstream execution
Tool input
A tool receives incorrect arguments
Upstream execution
Tool output
A tool omits information available from its source
Upstream execution
Appendix
Table 5: Evidence-chain stages used for failure attribution. The first unsupported stage determines the implicated repair owner.
Knowledge
False
Unknown
True
Atomic facts
5.6
15.3
4.2
Unstructured guide
9.7
26.4
5.6
Strategy package
5.6
15.3
4.2
− cond. rules
19.4
23.6
6.9
Appendix
Table 6: Strategy-condition violation rates (%) by condition outcome on 108 matched player-state variants.
Large language model (LLM) agents achieve state-of-the-art performance across domains through chain-of-thought (CoT) reasoning. Despite their ubiquitous use, little is understood about the relationship between the externalized trace and the model's actual computation. We address this by investigating how LLM agents make decisions in iterative games with incomplete information. Using a series of mechanistic interpretability techniques, we find that the model forms and iteratively revises an internal representation of the game throughout the CoT. Once the model prematurely commits to an incorrect belief, it rarely recovers. This failure is manifested in both the literal trace and the layer activations. Ultimately, as multi-agent systems are more prevalent, our work opens the door to a more focused diagnosis of their strategic reasoning errors. For illustrative purposes, we offer an interactive visualizer of our methodology on selected sample traces at https://interp-self.github.io/gametrace/
LLM-based game generation promises to turn natural-language specifications into executable games, but progress is limited by the lack of reliable automated verification. Unlike conventional code generation, game correctness is defined over long-horizon interaction: a game may appear correct while violating core mechanics such as state updates, interaction rules, and phase transitions. Existing Agent-as-a-Verifier approaches collapse verification into open-ended gameplay, making verdicts reachability-bound, time-consuming, coverage-limited, and sensitive to the agent's gameplay ability. We present GameGen-Verifier, an automated verification paradigm for LLM-generated games that decomposes a specification into verifiable keypoints and grounds them into independent verification units. Each unit patches the game runtime into a concrete target state, executes a bounded interaction, and judges the outcome against the keypoint assertion. We implement GGV-Harness, a scalable agentic harness providing concurrency management, runtime isolation, and fault recovery. On VeriGame, our dataset of 100 games across seven genres, GameGen-Verifier achieves up to 92.2% accuracy against human judgments versus 58.8% for the coverage-enforced Agent-as-a-Verifier baseline, while reducing wall-clock time by up to 16.6x.
We introduce State-Grounded Conditioning (SGC), a design principle for user-facing LLM agents that must condition on live user state (game state, session history, live inventory), and a distinct failure class we call direction drift: task-complete responses whose chosen direction misaligns with the current state. SGC externalises state-dependent control into rule kernels over structured inputs and three primary state slices, via Perception, Grounding, and Interaction wrappers with explicit conditioning dependencies. We evaluate SGC on a 200-session anonymised benchmark (≈1,000 assistant model turns) from an in-game conversational coaching agent that guides players through consecutive competitive matches, reporting mean first-token latency and five human-annotated dialogue-quality metrics that jointly cover factual grounding and coach-like guidance progression. The Perception wrapper holds mean first-token latency at 1.5s (vs. 6.1s for PE-Agent inside a production tool-use harness); enabling all three wrappers lifts turn-level grounded accuracy from 61.1%/69.8% (Prompting / PE-Agent) to 96.7% and session-level grounded accuracy from 20.0%/26.5% to 83.5%; session-level grounding-failure incidents drop by ≈78% relative to the strongest baseline. A cumulative ablation shows complementary incremental gains as the wrappers are added. These results inform approximate state-slice orthogonality, without establishing independent per-wrapper effects.