cs.LGNov 5, 2026
SaveDoes a Trading Policy Need to Know What It Trades? Microstructure Invariance and Zero-Shot Transfer of Execution Policies
Organizations: University of Massachusetts Amherst · University of Massachusetts Lowell
Abstract
Execution algorithms are calibrated asset by asset, because the queues, trade rates and bid-ask spreads they face differ by orders of magnitude across assets. For reinforcement learning this is zero-shot generalisation across contexts of very different scales, and microstructure invariance proposes a normalisation fixed by theory: measured in each asset's own units of time, size and price, markets look alike. In an exact order-by-order replay of public Nasdaq data, we train policies once, on 954 stocks in 2019, to choose between waiting in the queue and crossing the spread, and evaluate them unchanged on 1,964 unseen stocks in 2020-21, 2,629 stocks in 2025-26, crypto pairs and a second exchange. First, a policy that sees a stock only through its recent trade rate, trade size, spread and price increment (tick) costs within 0.015 spreads per share (a fifth of what learning saves) of a policy whose training data included the stock, and the stock's own history does not help it; it comes within a similar margin of policies trained on the second exchange when every episode counts equally, and within a wider one on crypto. Second, in mean cost before fees, invariant units beat raw units, z-scores and asset descriptors, but by under a basis point: most of their advantage is information and a common cost scale that a learner can also take as plain inputs, and none of the representations tested extrapolates to unseen spread regimes. Third, a near-zero transfer gap to a pooled comparator is weak evidence that a representation is right: we show that such a comparator detects only a small share of what a poor representation loses, and confirm it in synthetic markets and on pendulums.
Figures & tables
Figure 1: One policy for every asset? (a) The decision we study: a buy order resting at the best bid can wait or cross the spread. (b, c) Why units might suffice: the probability that a 100-share order joining the best bid fills within a horizon varies widely across 1,189 training stock-days when time is in seconds, and far less in each stock’s own mean time between trades (thick lines: medians within terciles of the relative tick, the price increment over the volatility per trade). (d) The test: policies trained once on 2019 Nasdaq stocks are evaluated unchanged on orange days; comparators trained on the test market use the grey days.
| Representation | Time | Size | Price | Cost | Inputs added |
| Invariant units | |||||
| MS | none | ||||
| MS+tick | tick ratios | ||||
| MS σ | none | ||||
| MS d | none | ||||
| KO | none | ||||
Table 1: Units of each representation. Features with dimensions are divided by the time, size and price units, and the policy is trained on the cost per share in the cost unit. , , and are the mean time between trades, trade size, spread and touch depth over the 30 minutes before the episode, is the volatility per trade, and , and are Kyle and Obizhaeva’s units ( Appendix A ). Tick ratios: the tick over the price unit and over , in logs.
Figure 2: Invariant units lead, but every transfer gap is near zero. (a) Each representation’s zero-shot cost minus that of MS units, on held-out stocks in 2020–21 (1,964 stocks) and 2025–26 (2,629 stocks). (b) Its transfer gap: zero-shot cost minus that of the same representation trained on as many episodes including the stock’s own 2019 days. Three seeds; 95% stock-bootstrap intervals; band: the margin.
Figure 3: The units supply mostly a cost scale and information, and they save data (exploratory). (a) Raw features’ cost relative to MS+tick on the held-out stocks, as the representation gains a cost in the stock’s spread ( raw s ) and then the window statistics the units are built from ( raw+info ), for fitted Q-iteration with trees and with a neural network; 95% stock-bootstrap intervals; the band is the margin. (b) Validation cost against the number of training episodes from other stocks, relative to MS+tick with the full training set; dashed lines: a policy trained on each stock’s own five training days, and posting and waiting.
Figure 4: The stock-trained policies transfer to crypto and to a second exchange. Paired cost differences; bands: the margins; arrows: intervals beyond the axis. (a) Coinbase pairs, in volatility per trade, 90% intervals. (b) The ten IEX days after the policies were frozen, books at one tick, in spreads per share, 95% intervals, with every stock or every episode weighted equally. FQI: fitted Q-iteration.
Figure 5: Wrong units cost more without a larger transfer gap, unless the comparator can tell the new assets apart. For each representation, its zero-shot cost minus that of the invariant units (filled) and its transfer gap (open). (a, b) Synthetic order-book markets, exactly similar in MS units, with test assets inside and outside the training scales; spreads per share. (c, d) Pendulums, exactly similar in natural units, with test systems inside and outside the training range of time scales; cost per step. 95% bootstrap intervals over test assets or systems.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
| fitted Q-iteration, trees (main) | fitted Q-iteration, neural (exploratory) | PPO | |
|---|---|---|---|
| model | LightGBM regression, one per decision | MLP, 3 hidden layers of 256 SiLU units, one per decision | policy and value MLPs, 2 hidden layers of 128 SiLU |
| settings | 300 rounds, 63 leaves, learning rate 0.05, samples per leaf, feature fraction 0.9, bagging 0.8, L2 penalty 1, deterministic | AdamW, learning rate , weight decay , batch 4,096, up to 40 epochs, early stopping (patience 4) on 5% of episodes held out | clip 0.2, GAE , no discounting, 4 epochs of minibatches of 16,384, Adam , entropy bonus 0.01 to 0.001, Huber value loss, gradient norm clipped at 1, advantages standardised |
| inputs | the representation’s features | , standardised | as the neural learner, plus the decision index |
| target | advantage over crossing now, squared error | the same, scaled to unit RMS per decision | the episode cost (policy gradient) |
| data | 376,610 episodes, all five actions branched at each decision; behaviour policy hold 40%, join 25%, improve 10%, cancel 10%, cross 15% | the same | 150 iterations, each on 35% of the stock-days, about 130,000 episodes per iteration |
| seeds | 3 (8 in some analyses) | 5 | 3 |
Table C.1: The learners. Episodes: the training stocks’ five 2019 days (4,431 stock-days, 100 start times each, before the horizon and closing-time rules). Times: wall clock on a shared machine.
| default | 1,000 rounds | 255 leaves | both | |
|---|---|---|---|---|
| cost relative to the arm’s default setting | ||||
| MS+tick | 0 | [ , ] | [ , ] | [ , ] |
| raw | 0 | [ , ] | [ , ] | [ , ] |
| raw s | 0 | [ , ] | [ , ] | [ , ] |
| raw+info | 0 | [ , ] | [ , ] | [ , ] |
| minus MS+tick at the same setting | ||||
Table C.2: Capacity of the tree learner (validation, exploratory). Top: each arm’s cost relative to its default setting; bottom: each arm minus MS+tick at the same setting. Spreads per share, stocks weighted equally, 95% intervals (stock bootstrap stratified by relative-tick tercile). One seed per larger setting, three for MS+tick with 1,000 rounds; the default column averages its three seeds, so seed variation (a standard deviation of about 0.004 for a validation contrast) is not in the intervals of the other columns.
Figure E.1: Synthetic controls (zero-intelligence market, advantage learner; 40 test assets, 95% cluster-bootstrap intervals). Circles: transfer gap of each representation, zero-shot minus a policy that also saw the test assets. Diamonds: zero-shot cost minus the zero-shot cost in microstructure ( MS ) units. Wrong units cost more in every control, but their transfer gap stands out only when the test scales lie outside the training range (b).
| Representation | Cost | Minus MS | Transfer gap |
|---|---|---|---|
| Invariant units | |||
| MS | 0.439 | reference | [ , ] |
| MS+tick | 0.432 | [ , ] | [ , ] |
| MS σ | 0.442 | [ , ] | [ , ] |
| MS d | 0.438 | [ , ] | [ , ] |
| KO | 0.440 | [ , ] | [ , ] |
Table F.1: Stage 1 (314,133 episodes of 1,964 stocks, 2020–21). Zero-shot cost in spreads per share (three seeds), its difference from MS units, and the transfer gap on the 1,001 stocks whose 2019 days the seen policy saw; 95% stock-bootstrap intervals. n/a: no seen policy was trained in raw s units. The baselines cost 0.642 (crossing at once) and 0.508 (posting, then crossing).
| Representation | Cost | Minus MS | Transfer gap |
|---|---|---|---|
| Invariant units | |||
| MS | 0.477 | reference | [ , ] |
| MS+tick | 0.474 | [ , ] | [ , ] |
| MS σ | 0.479 | [ , ] | [ , ] |
| MS d | 0.477 | [ , ] | [ , ] |
| KO | 0.476 | [ , ] | [ , ] |
Table F.2: Stage 2 (1,256,793 episodes of 2,629 stocks, 2025–26), as Table F.1 ; transfer gaps on the 737 stocks the seen policy saw. n/a: no seen policy was trained in raw s units. The baselines cost 0.665 (crossing at once) and 0.522 (posting, then crossing).
Figure F.1: Without tick context, losses concentrate where the tick binds. The 2020–21 contrasts and transfer gaps of Figure 2 for all stocks and for each tercile of the relative tick. (a) Zero-shot cost minus that of MS units; (b) transfer gap. Three seeds; 95% stock-bootstrap intervals; band: the margin.
Figure F.2: Stage 2 (2025–26) by tercile of the relative tick , as Figure F.1 . The contrasts are smaller than in 2020–21.
| Contrast | Val. | Stage 1 | Stage 2 | Crypto |
|---|---|---|---|---|
| Neural fitted Q-iteration | ||||
| raw MS | [ , ] | [ , ] | [ , ] | [ , ] |
| raw+desc MS | [ , ] | [ , ] | [ , ] | [ , ] |
| z-score MS+tick | [ , ] | [ , ] | [ , ] | [ , ] |
| KO MS | [ , ] | [ , ] | [ , ] | [ , ] |
| MS+tick MS | [ , ] | [ , ] | [ , ] | [ , ] |
Table F.3: Learners and learned normalisation. Paired cost differences in spreads per share (crypto: volatility per trade), seeds averaged, with 95% intervals that resample stocks and training seeds together. Learned scale: each training stock’s reward divided by a scale learned from its own data (PopArt-style); random rescaling: training prices and sizes rescaled at random over a factor of 100; descriptors: the four inputs that raw+desc adds.
| Contrast | Gross | Net of fees | Notional-weighted |
|---|---|---|---|
| Validation (2019) | |||
| raw MS | [ , ] | [ , ] | [ , ] |
| raw s MS | [ , ] | [ , ] | [ , ] |
| raw raw s | [ , ] | [ , ] | [ , ] |
| MS+tick MS | [ , ] | [ , ] | [ , ] |
| MS+tick post, then cross | [ , ] | [ , ] | [ , ] |
Table F.4: Economic size. Paired cost differences in basis points of the price: gross, net of Nasdaq’s fees, and with stocks weighted by notional instead of equally; 95% stock-bootstrap intervals.
Figure G.1: The pendulum family (exploratory). (a, b) Trees: transfer gap (zero-shot minus seen, blue) and zero-shot cost minus that of natural units (orange) for each representation, in and out of range; open circles in (b) are seen policies with all out-of-range systems pooled at half their data (design (d)). Natural units cost 0.602 per step. (c) Trees: transfer gap against the test system’s share of the seen policy’s data (log scale), for raw and natural units, over 9 seeds; the dashed line and band are raw ’s zero-shot excess over natural units on the same systems. (d) raw+info minus natural units, zero-shot, in range (filled) and out of range (open), for trees, trees with four times the data, and the neural learner. 95% bootstrap intervals over test systems, stratified by seed; in (c) they also resample seeds.
Figure G.2: CARL’s pendulum (exploratory). Transfer gap (zero-shot minus seen, blue) and zero-shot cost minus that of nat+ctx (orange) per representation; top trees, bottom the neural learner. (a, d) In-range test contexts, each 1/72 of a seen policy’s data; (b, e) contexts whose lie out of range but whose problem does not; (c, f) every feature out of range (slower, lighter pendulums); out of range the seen policy pools all such contexts, each 1/32 of its data. Gym’s cost per step; in range nat+ctx costs 0.908 (trees) and 0.915 (neural), the oracle 0.779. 95% bootstrap intervals over seeds, then contexts.