A deployed LLM coaching agent answers requests whose correct response depends on player state and game version, so advice that is right for one player can be wrong for the next. We describe TactiNavi, the serving system behind the LLM coach of a commercial open-world game, which makes strategy applicability an explicit artifact shared by offline compilation, online execution, failure diagnosis, and release. An offline compiler turns versioned game evidence and expert guidance into strategy packages with explicit prerequisites, the serving plane evaluates those prerequisites as three-valued conditions over structured player state and records decision traces, and a control plane uses the resulting condition-rule dependencies to validate revisions on held-out traffic before release. In a retrospective evaluation of 1,122 logged coaching turns from three deployment periods, condition-bearing packages raise correctness from 57.9% to 82.0% over atomic facts and reduce strategy misapplication under changed player state from 18.1% to 10.4%, while trace-based claim verification reaches 86.4% precision at a median reply latency of 2.07 s. The deployment also showed that most apparent fixes are generation noise, because an unchanged-system rerun recovers 38.4% of failing turns and only one of 15 shipped revision units passes the release checks retrospectively. We distill these results into lessons on treating applicability as a first-class serving object, sizing traces for later diagnosis, and building the null model into the release path.
Figures & tables
Req.
Mechanism
R1
Versioned packages with evidence, prerequisites, and applicable versions (Sec. 4.2 ).
R2
Three-valued conditions over state, evaluated before generation (Sec. 4.3 ).
R3
Traces and condition–rule dependencies for attribution (Sec. 4.4 ).
R4
Scoped validation, counterexamples, and release gates (Sec. 4.5 ).
Table 1: Serving requirements and the mechanisms that address them.
Figure 1: Overview of TactiNavi. The Strategic Knowledge Compiler supplies versioned strategy packages for online condition evaluation. The Agent Execution Plane generates guidance and records decision traces, which support feedback-driven diagnosis. Attributed repair targets enter the Evolution Control Plane for independent validation, and accepted revisions update knowledge and execution components for subsequent coaching.
Knowledge
Corr.
Contr.
Q2
Q3
Pass
Tok.
Viol.
Atomic facts
57.9
0.91
1.02
1.21
41.9
767
10.4
Guide
82.5
0.28
1.27
1.43
62.7
1298
18.1
Package
82.0
0.28
1.32
1.44
65.2
1477
10.4
− cond. rules
75.6
0.43
1.19
1.43
61.4
1246
21.5
Table 2: Knowledge configurations on T3 strategic turns. Corr., Pass, and Viol. are percentages. Viol. is measured on false and unknown variants. Contr. counts contradicted claims and violations. Q2 and Q3 range from 0 to 2. Tok. counts knowledge tokens.
Method
Prec.
Rec.
Rej.
Lag.
Doubt as failure
75.0
100.0
0/12
0.0
Direct feedback
75.6
94.4
1/12
0.0
Category routing
88.0
61.1
9/12
0.0
Who&When
77.8
97.2
2/12
0.0
TactiNavi
86.4
52.8
9/12
26.7
− turn localization
80.8
58.3
7/12
0.0
Table 3: Failure diagnosis on expert-adjudicated T1 doubts. Prec. and Rec. are percentages for founded doubts. Rej. counts rejected unfounded doubts out of 12. Lag. is delayed-feedback localization (%).
Revision
Units
Rec.
Gain
Spec.
Eligible
Unchanged rerun
–
38.4
+0.6
–
–
Deployed units
15
–
–
–
1/15
Planted, entity
3
39.1
−0.6
3/3
0/3
Planted, mechanism
3
40.7
+0.7
0/3
0/3
Table 4: Independent revision assessment on T3 . Rec. is recovery among previously failing turns for the unchanged rerun and among failing exposed turns for planted controls (%). Gain is the paired correctness difference in percentage points, evaluated in scope for planted controls. Spec. counts case-specific revisions, and Eligible counts those passing release checks. Unchanged reruns use S1 , while planted controls use S3 .
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Stage
Unsupported when
Repair owner
Reply rendering
The response misstates or exceeds its reply context
Reply execution
Context assembly
Available relevant evidence is omitted from the reply context
Reply execution
Result merging
A correct result is lost when tool outputs are combined
Upstream execution
Routing
A required tool or knowledge source is not selected
Upstream execution
Tool input
A tool receives incorrect arguments
Upstream execution
Tool output
A tool omits information available from its source
Upstream execution
Appendix
Table 5: Evidence-chain stages used for failure attribution. The first unsupported stage determines the implicated repair owner.
Knowledge
False
Unknown
True
Atomic facts
5.6
15.3
4.2
Unstructured guide
9.7
26.4
5.6
Strategy package
5.6
15.3
4.2
− cond. rules
19.4
23.6
6.9
Appendix
Table 6: Strategy-condition violation rates (%) by condition outcome on 108 matched player-state variants.