Staged LLM agents make market and social simulators realistic, but one 102-day run of a 200-agent, ten-index market simulator issues 203,100 language-model calls and returns one realization. Conventional low-compute distillation drops their free-form intermediate state. We present staged semantic distillation, a low-compute method that preserves non-structured semantics in a non-LLM student trained from one recorded trace. A frozen encoder maps each teacher output to a 1024-d vector; the student sees typed stage-tagged vectors with stage-specific visibility, is jointly supervised, and trains with scheduled self-history. One TwinMarket trace yields a 13.2M-parameter deterministic student that trains in 1,658 seconds on one 8-GB GPU and runs with zero teacher or decoder calls and no text generation (0.14 ms per decision). Its analysis stays close to the teacher (cosine 0.943), and swapping semantic inputs between agents moves 64.7% of decisions against 12.8% for a matched donor, so the channel is preserved and live. We use the surrogate to study what the transfer carries. It reaches 0.8347 balanced accuracy on the teacher-forced replay and 0.7783 [0.766,0.791] on the deployment decoder; the five-seed gain over a no-semantic retrain is +2.2 [1.21,3.14]. Against the teacher's own re-sampling level, round-one R is 97.1% [95.8,98.4] and 92.3% [89.0,95.4]; R is a diagnostic, not an accuracy. A five-seed content-by-schedule study isolates the gain: scheduled self-history carries it (+2.2 [1.21,3.14]); shuffled and constant content controls reproduce it, while content matters teacher-forced. The design rule: invest in the deployment schedule, let content earn its place at the teacher-forced operating point. A 20-seed, 200,000-decision sweep takes 12 minutes with zero LLM calls, versus three hours for the teacher. The paper is a method-and-case-study contribution on one simulator and trace.
Figures & tables
Figure 1: Staged semantic distillation. Left: the teacher’s staged policy writes free-form outputs at five semantic stages and then acts; the stages have different visibility, and the recorded text is the intermediate state to preserve. Middle: each output is encoded once by a frozen text encoder into a 1024-d vector and exposed to the student as a typed, stage-tagged token with a visibility mask. Right: a 13.2M-parameter Transformer is trained with joint stage supervision and scheduled self-history and runs deterministically with no text generation and no teacher or decoder calls at inference.
Information set
Parity bal.
Parity acc.
A: recorded analysis
0.9368
0.9370
B: fresh analysis
0.7860
0.7863
Table 1: Teacher re-sampling level on the round-one test decisions ( n=1,998 ; 20 replays per context), in balanced-accuracy and accuracy space; the balanced 95% cluster-bootstrap interval is [0.9296,0.9448] for A and [0.7729,0.7998] for B. Condition A is the denominator of the teacher-forced (P1) fidelity ratio; condition B the denominator of the round-one self-analysis ratio.
Quantity
Value
Diagnostic
Note
Deployment (P2, full test window)
Deployed (5-seed mean)
0.7776
–
0.7712–0.7825
No-semantic retrain (5 seeds)
0.7552
–
Deployment gain
+2.2
[1.21,3.14]
rounds 2–5
Schedule removal (P2)
5.9 / 2.3 / 3.2
–
Table 3
Round-one diagnostic (not a deployment predictor)
Table 2: Primary results. The top block is the deployment result (P2 within-day autoregressive decoder) and the five-seed recipe contrast; it is what the deployment claims rest on. The bottom block is the round-one sample-fidelity diagnostic: R is the student’s balanced accuracy relative to the teacher’s own re-sampling level, not an accuracy—a constant predictor reaches 53.4%/63.6% on the same scale and R can exceed 1. No later-round teacher replay exists, so the deployment decoder has no R ; the schedule/content decomposition is in Table 3 .
Figure 2: (a) Teacher re-sampling level (open diamond) and deployed-student balanced accuracy (filled circle) for the two information sets; the self-analysis row also shows decomposition references (fresh heads on the same generated vector, an HGB read-out of the generated analysis, and a state-only HGB baseline), which consume rather than generate the analysis vector. (b) Reliability of the deployed side probability against the re-sampled teacher probability (Brier 0.033 with the teacher’s analysis, 0.082 with its own). (c) Closed-loop daily excess kurtosis over the 102-day window, mean ± sd across three simulator seeds; the index set is labelled and sampling collapses the tails.
Content
Schedule on
Schedule off
Difference
Agent-matched
0.7783
0.7193
5.9
Within-date shuffled
0.7816
0.7591
2.3
Constant
0.7857
0.7534
3.2
Table 3: Content-by-schedule design on the within-day deployment decoder (five deployed-recipe seeds, three seeds per control cell). The constant row fixes only the semantic-history content; the current context, persona/news/state inputs and, with the schedule on, the action-history schedule remain, so it is not an empty-input control. Removing the schedule is the active ingredient in every content cell; with the schedule on, content changes the decoder by at most 0.74 points (intervals include zero).
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Comparison
Rows
Excluded
Round-one labelled decisions
1,998
2 no executed order
Self-analysis
1,982
16 incomplete agent-day
Donor probe
1,995
3 no donor pair
One-step P1, full window
9,956
44 no executed order
Autoregressive P2, full window
9,910
18×5 slots, 46 labelled
Round-one train / val.
17,058 / 1,197
142 / 3 no order
Appendix
Table 4: Row flow for every denominator in the paper. “Excluded” gives the number of slot rows and the reason. The 1,982 self-analysis rows are the round-one rows of the 1,982 complete agent-days used by the autoregressive decoder, so the two comparisons share that row set.
Student
K
Bias (pts of R )
95% CI width
Coverage
Bayes-optimal
5
−0.01
0.205
1.000
20
−0.15
0.185
1.000
100
−0.19
0.176
1.000
Flipped 10%
5
−0.52
0.197
0.985
20
−0.06
0.176
1.000
100
+0.02
0.171
0.995
Appendix
Table 5: Synthetic-teacher validation of the sample-fidelity ratio. 1,000 contexts, p∼Beta(5,5) ; 200 outer replicates per cell; two-level bootstrap with 300 replicates. Bias is the mean estimated minus true ratio, in percentage points of R ; the true ratio is about 1.37 for the Bayes student and 1.31 for the flipped one. Coverage is the fraction of 95% intervals containing the true ratio.
Stage
P
N
Q
V
S
A
C
H
R
M
D
T
News query
∙
Indicator query
∙
∙
Indicator summary
∙
∙
∙
Day analysis
∙
∙
∙
∙
∙
Rationale
∙
∙
∙
∙
∙
∙
∙
∙
∙
Action
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
Appendix
Table 6: Stage visibility matrix of the deployed student. Columns: P persona, N retrieved news (two tokens), Q indicator query, V indicator values, S indicator summary, A day analysis, C account state and extra context, H previous-round parameters, previous actions, competition and day-cumulative features, R current-day round history, M semantic history (analysis and rationale of earlier rounds), D previous-day history, T current rationale. Filled cells are the token groups each stage’s attention prefix admits.
Arm
Seed
P1
P2
r1
Deployed
0–4 mean
0.8311
0.7776
0.7339
No-semantic
0–4 mean
0.7848
0.7552
0.7522
Shuffled
seed 0
0.7818
0.7816
0.7254
Shuffled
seed 1
0.7874
0.7847
0.7483
Shuffled
seed 2
0.7879
0.7801
0.7347
Shuffled
mean
0.7857
0.7821
0.7361
Appendix
Table 7: Seed-level content controls. P1 = full-test teacher-forced one-step balanced accuracy (9,956 rows); P2 = full-test within-day autoregressive balanced accuracy (9,910 rows); r1 = round-one self-analysis balanced accuracy (1,982 rows). The deployed and no-semantic rows give the five-seed mean; the control rows give individual training seeds and their three-seed mean. The shuffled control uses another agent’s recorded same-date/round analysis/rationale with a fixed within-split permutation; the constant-content control sets every semantic-history value to the training mean and is evaluated content-free.
Operating point
Brier
TV
KL
ECE
NLL
Mode
Deployed, teacher analysis
0.033
0.088
0.116
0.013
0.206
0.931
Deployed, self analysis
0.082
0.219
0.234
0.069
0.552
0.770
Interface, teacher analysis
0.027
0.086
0.098
0.019
0.189
0.945
Appendix
Table 8: Distributional fidelity of the student’s side probability to the re-sampled teacher conditional (round-one test; 20 replays per context). Brier is binary; TV is total variation; ECE is ten-bin expected calibration error against p^ ; NLL is the negative log-likelihood of the recorded action; mode is the fraction of contexts with the same most likely action as the re-sampled teacher. Date × asset cluster-bootstrap 95% intervals for the deployed rows, teacher-analysis / self-analysis: Brier [0.028, 0.040] / [0.072, 0.094]; TV [0.079, 0.097] / [0.202, 0.237]; KL [0.098, 0.135] / [0.209, 0.262]; ECE [0.010, 0.025] / [0.047, 0.093]; NLL [0.181, 0.234] / [0.516, 0.589]; mode [0.918, 0.943] / [0.735, 0.803]. A constant base-rate forecast attains Brier 0.219, TV 0.454, KL 0.600 and mode 0.554 on the teacher-analysis rows.
Variant
One-step bal.
AR bal.
Closed-loop kurt.
No self-history schedule
0.8143
0.7193
–
Shuffled content, no schedule
0.7907
0.7591
–
Constant content, no schedule
0.7841
0.7534
–
Action-only BC (analysis sch.)
0.8226
0.7324
–
Analysis-only schedule
0.8218
0.7419
2.30 ± 0.58
Full sched., no stage losses
0.8265
0.7660
–
Appendix
Table 9: Design study on the full test window (one-step n=9,956 ; within-day autoregressive n=9,910 ). All variants share the data, split and evaluation protocol; balanced accuracy is reported. The last column gives the closed-loop tail level for the variants that were run in closed loop (Section 5 ); those rows are ten-index, and the ten-index teacher value is 0.90 (the six-index teacher value is 0.66). The deployed model is the full-schedule variant with independent stage heads and the full action representation. The action-only row is the behaviour-cloning control: all five semantic stage losses are set to zero and only the analysis slot is scheduled. With the full-schedule/no-stage-loss row it completes a 2 × 2 over schedule extent (analysis-only versus full) and stage supervision (on/off), reported in the text; the cells are one training seed each. The no-semantic row is seed 0 of a five-seed retrain with the semantic-history input masked during training and evaluation (Section 4 ); its five-seed full-test values are 0.7842/0.7805/0.7919/0.7896/0.7778 and 0.7498/0.7561/0.7659/0.7535/0.7508. The shuffled-content row keeps the same slots, masks and schedule but takes every recorded analysis/rationale vector from another agent on the same date and round (three training seeds), so it changes the agent-to-vector identity link while holding the vector distribution and the date-specific action composition fixed. The content-free row keeps the slots, masks and schedule but sets every semantic-history value to the training mean (zero in normalised space) and is evaluated content-free (three training seeds; Appendix A.6 ); per-seed values are in Table 7 . The two no-schedule content rows repeat the shuffled and constant-content cells with all three scheduled-sampling probabilities set to zero (one training seed each); with the matched-content/no-schedule row they form the 2 × 3 content-by-schedule design of Table 3 . The flat-token row has the same input features and a matched parameter budget but no stage tags, visibility, stage heads or stage losses, so it is one-step only (it cannot generate stage outputs). The flat MLP is an action-only behaviour-cloning model on the numeric context with no semantic channel; the HGB row is the state-only structured-feature baseline, also one-step. The flat-token model sees the recorded semantic inputs while the no-semantic retrain does not, so their gap (3.35 points) is an information-set and depth difference, not a typing effect; the typing-only contrast is the flat-token row against the staged deployed row at the same inputs (1.70 points).
Decoder / model
R1
R2
R3
R4
R5
One-step, deployed
0.9098
0.8456
0.7949
0.8222
0.8007
One-step, retrain
0.7470
0.8137
0.7807
0.7807
0.7914
One-step, second seed
0.9137
0.8289
0.7753
0.7920
0.8189
Autoregressive, deployed
0.7252
0.8196
0.7752
0.7961
0.7740
Autoregressive, retrain
0.7485
0.7502
0.7588
0.7436
0.7407
Autoregressive, second seed
0.7348
0.8110
0.7660
0.7779
0.7866
Appendix
Table 10: Per-round balanced accuracy ( n=1,982 per round on the autoregressive decoder; 1,989–1,998 on the one-step decoder). The retrain never receives semantic history; it is insensitive to the analysis slot by construction, so its one-step and self-analysis numbers are the same baseline. Per-round entries are unweighted means of round-specific balanced accuracies; the pooled full-test values reweight rounds by class frequency, which is why the pooled value differs from the row mean by 0.14–0.19 points for the retrain and the second seed and by 0.03 (autoregressive) or 0.006 (one-step) for the deployed model.
Series
Excess kurt.
Max move
Vol (bp)
ARCH(1)
Real market (10)
1.66
4.727%
137.2
−0.05
Teacher path † (6)
0.66
5.15%
168.3
0.13
Teacher path (10)
0.90
5.97%
191.0
0.13
Deployed det. (6, 3)
0.38 ± 0.18
4.727%
162.1
0.06
Deployed samp. (6, 3)
0.03 ± 0.30
3.87%
139.7
0.04
Deployed det. (10, 3)
0.68 ± 0.11
4.766%
156.9
0.07
Appendix
Table 11: Closed-loop macro statistics over the 102-day window. Rows marked (6) use the six indices of the teacher reference and rows marked (10) are means over ten indices; student rows additionally show the number of simulator seeds and report mean ± sd across seeds (each seed is the mean over its index set). Kurtosis and volatility show the seed mean ± seed sd; maximum move and ARCH(1) are seed means. The teacher path is one recorded realization, not a distribution over teacher runs. The same checkpoint, initialisation and window are used across both decoding rules; the decoding rule and the simulator seed change across rows, and within a decoding rule only the seed changes. † Teacher recorded path: mean over the six indices whose end-of-window level stays within 40% of the real index and whose volatility stays within 60% of it (CPEI, EREI, FSEI, MEI, TSEI, TTEI); the other four paths show a strong end-of-window level drift. The all-ten rows are primary and the six-index rows secondary/outcome-selected. The analysis-only and tied+no-bypass entries are design variants. ‡ The real-market maximum (4.727%) and the deployed six-index maximum (4.727%) coincide to three decimals because both are means of per-index maxima over different index sets: the real-market figure is the ten-index mean (4.7273%), while the deployed six-index figure is the mean over its three seeds of the six-index means (4.6365/4.7723/4.7714, i.e. 4.7267%); they are different series. The ten-index values are 5.97% (teacher) and 4.766% (deployed). Volatility in basis points; ARCH(1) is the lag-1 autocorrelation of squared daily returns.
Read-out
Balanced accuracy
State only (HGB)
0.7424 [0.713, 0.770]
Analysis only (HGB, deployed H)
0.7516 [0.721, 0.777]
Analysis only (logistic, deployed H)
0.7567 [0.730, 0.779]
Teacher analysis only (HGB)
0.9076 [0.894, 0.921]
Re-trained head, 4,866-param linear (3 seeds)
0.7425 ± 0.0061
Re-trained head, 2,433-param logistic
0.7439 [0.717, 0.768]
Appendix
Table 12: Encoding read-out, head re-training and self-consistency (round one). Reader rows: HGB or logistic on the student’s state features (checkpoint-independent), on the deployed checkpoint’s own generated analysis vector, and on the frozen-backbone head features [act;a;r] with a head re-trained on the 17,058 training rows and evaluated on the full 1,998 test rows; the tabulated head values are on the matched 1,982 rows (256-unit MLP 0.7468 ± 0.0106, 4,866-parameter linear refit 0.7425 ± 0.0061, 4,872-parameter MLP 0.7483 ± 0.0123, 2,433-parameter logistic 0.7439; the 4,872-parameter MLP has two hidden units, i.e. 2,432×2+2=4,866 parameters into the hidden layer plus 2×2+2=6 into the output layer). The family’s three-seed reader sd is about 0.4 points. All reader values are balanced accuracy with 95% date × asset cluster-bootstrap intervals where available. Self-consistency: two own passes, action with an empty analysis slot versus action after injecting the student’s own analysis, on 1,998 round-one rows.
Read-out
Features
Rows
Bacc
HGB
teacher analysis vector (1024-d)
1,998
0.9076 [0.894, 0.921]
HGB
teacher → generated vectors
1,998
0.7421 [0.712, 0.768]
HGB
generated → teacher vectors
1,998
0.7380 [0.710, 0.763]
HGB
structured + teacher analysis
1,998
0.9044
Logistic
teacher analysis text
1,998
0.9205
Logistic
teacher analysis vector (1024-d)
1,998
0.9180
Appendix
Table 13: Teacher-analysis read-outs on the round-one test rows. These controls come from separate runs with different readers and feature sets and are not recomputed under one protocol. The HGB on the teacher’s recorded analysis vector is the primary baseline for the 15.6-point content gap; the structured-plus-analysis HGB is the full-information-set fair-baseline comparison; the cross rows train the same HGB on one analysis-vector source and evaluate it on the other; the logistic text-versus-vector pair is the frozen-encoder control.
Recipe
n
One-step
AR
Round-one
Deployed recipe
5
0.8311 ± 0.0051
0.7776 ± 0.0041
0.7339 ± 0.0113
No-semantic retrain
5
0.7848 ± 0.0060
0.7552 ± 0.0065
0.7522 ± 0.0023
Analysis-only schedule
3
0.8239 ± 0.0034
0.7349 ± 0.0115
–
Interface checkpoint
1/3
0.8192 (seed 0)
–
0.7391 ± 0.0291
Appendix
Table 14: Training-seed replication of the four recipes used in the comparisons. Balanced accuracy on the full test window (one-step n=9,956 ; AR = within-day autoregressive, n=9,910 ) and on the 1,982 round-one self-analysis rows, mean ± sd across the seeds. The interface checkpoint has no autoregressive run; its n entry is one-step/round-one seeds (1/3), and its round-one row is the matched-row value. The deployed checkpoint is seed 0 of the five deployed-recipe seeds.
Statistic
Mean
SD
Min
Max
Excess kurtosis
−0.29
0.28
−0.74
0.23
Volatility (bp)
146.9
7.5
134.1
162.9
Max daily move (%)
3.15
0.22
2.89
3.71
Appendix
Table 15: Twenty-seed capability sweep on the ten-day held-out window (2023-11-02..2023-11-15): 200 deterministic closed-loop runs (ten indices × twenty simulator seeds). Each seed contributes the mean over the ten indices; columns are mean, sd, min, and max across the twenty seed-level means. The ten-index teacher recorded path on the same window is −0.48 excess kurtosis, 139.8 bp volatility, and 3.00% maximum move (date-block bootstrap intervals [−1.15,0.27] , [101,146] bp; the max-move interval [2.15,3.00]% is bounded above by the observed maximum by construction and is not used as a pass/fail check); this is a single recorded teacher path on the same window.
Figure 3: Compute–fidelity frontier. The same recipe is trained at student widths 192, 256, and 384 (5.0M, 7.4M, 13.2M parameters); points are full-test balanced accuracy for the one-step teacher-forced replay and the within-day autoregressive rollout, with 95% date × asset cluster-bootstrap intervals. The autoregressive point estimates are monotone in width, but the intervals overlap; the one-step point estimates are not monotone and also overlap.
Teacher run
Student
LLM calls
203,100
0 after the trace
Prompt tokens
691.5M
0
Trace and encoder
produced by the run
one-time input, 1 pass
Training
–
1,658 s, 8-GB GPU
First 102-day run
93 min
trace + 1,658 s + 2 min
Re-run, 102 days
93 min
≈ 2 min
Appendix
Table 16: Deployment economics. The first run of the surrogate is not cheaper than a teacher run: it consumes the recorded trace (203,100 calls, 691.5M prompt tokens, about 93 min), one frozen-encoder pass over the trace text, 1,658 s of student training and about 2 min of simulation. The amortisation is on re-runs: each further 102-day realization costs about 2 min and no LLM calls. Student counts exclude the frozen 0.6B embedding encoder, which is applied once, offline, and is not in the decision loop. The 119,880 replay calls are evaluation-only and are not in any column. New text outside the pre-encoded corpus would need one encoder pass per stage output, so the zero-call row is an inference-loop statement.
Language model distillation transfers a larger model’s capabilities to smaller or specialized models using outputs, probabilities, or internal representations.