A small fine-tuned parser that emits a well-formed but wrong tree fails silently: nothing downstream can tell. We ask whether the output format, a free design choice, controls how often this happens. On the PIZZA ordering benchmark we fine-tune two sub-billion models (Qwen3.5-0.8B and LFM2.5-350M) with LoRA on six lossless encodings of the same target tree, from 44.3 down to 10.9 tokens per order, including two matched-length pairs that vary surface form at fixed length. Across 300 runs (5 training pool sizes, 5 seeds) with a hierarchical bootstrap and Holm correction fixed before the results were read, shortening alone never helps, but writing the target in ordinary words does. The word encoding is the most accurate format on both models (87.5% vs. 85.8% exact match for the native s-expression on Qwen) while emitting 4.1x fewer tokens, and it cuts the silent error rate by 2.3x on Qwen and 1.8x on LFM2, separating at every pool size. Error analysis locates much of the gain in negation. All runs also learn to refuse off-menu input (held out: 95.9-99.0% handled) and to apply in-utterance self-corrections. Strict refusal has a measured price: 2.1-10.1% of real, mostly polite, orders are refused.
Figures & tables
Format
Tok.
Target
exr
38
(ORDER (PIZZAORDER (NUMBER 1 ) (COM...
json
33
{"orders":[{"type":"p","attrs":[{"n...
exr-short
25
(O (P (N 1) (C (Q EXTRA) (T CHEESE)...
tagged
13
pizza n=1 extra=cheese topp=pepperoni
dsl
8
p 1 +cheese pepperoni
words
7
pizza 1 extra cheese pepperoni
Table 1: One test order, “i want a pie with pepperoni and extra cheese” , in the six encodings (long targets truncated). Tokens are Qwen’s.
Figure 1: The order of Table 1 under Qwen’s tokenizer. For each format the upper bar has one box per token and the lower bar draws the same tokens as wide as their characters (both bars to a common scale across formats). Bottom: the matched pair with its tokens; words is longer as text and shorter in tokens. Boundaries come from the tokenizer’s own merge table.
Format
Tok.
A (EM) ↑
S (silent) ↓
Dparse
Drefuse
Inv./utt.
ms
Qwen3.5-0.8B
exr
44.3
85.8 ± 1.6
10.3 ± 0.8
0.6 ± 0.8
3.3 ± 1.1
0.100
1171
json
44.1
81.6 ± 1.3
14.9 ± 1.3
0.3 ± 0.3
3.2 ± 1.0
0.126
1220
exr-short
32.3
75.2 ± 3.6
17.4 ± 4.1
3.3 ± 2.0
4.2 ± 2.2
0.140
903
tagged
22.2
85.9 ± 2.9
11.6 ± 2.4
0.5 ± 0.3
2.1 ± 0.8
0.095
616
dsl
11.3
79.7 ± 1.9
8.3 ± 2.3
6.7 ± 3.2
5.4 ± 2.3
0.070
325
Table 2: Test results at the full pool (2,456,443), mean ± seed SD over 5 seeds, in percent of the 1,357 human-written test orders. A + S + Dparse + Drefuse = 100. Bold: best format in the column; underlined: not separable from the best (Holm, p≥0.05 ). Tok.: mean output tokens. Inv./utt.: invented entities per order. ms: median latency at batch size one on one GPU, mean over seeds.
Figure 2: Exact match against output tokens at the full pool, 95% bootstrap intervals. Dashed lines join the matched-length pairs ( exr / json , dsl / words ).
Difference (points)
1k
10k
100k
1M
2.5M
Qwen3.5-0.8B
words − dsl
+6.7 †
+4.8 †
+6.2 †
+5.8 †
+7.8 †
exr − json
+9.3 †
+3.8
+3.6
+5.5 †
+4.2 †
exr − exr-short
+6.2 †
+11.6 †
+10.3 †
+9.3 †
+10.6 †
words − exr
+4.0
+1.8
+1.6
+0.0
+1.8
S : exr − words
+11.4 †
+5.1 †
+4.4 †
+5.0 †
+5.8 †
Table 3: Paired differences in exact match (points; last row: silent rate, positive = words lower) at every pool size, both models. † : separates (Holm within the row). 95% intervals in Appendix Table 6 .
Figure 3: Where the test orders go at the full pool (axis starts at 60%). Labels: silent error rate S .
Figure 4: Silent errors by kind at the full pool, as a share of all test orders (bar height = S ). Negation: a NOT attribute added, lost or misplaced; line count: wrong number of pizzas or drinks.
Format
Neg. handled ↑
Real refused ↓
Corr. EM ↑
Corr. applied
Probes /40
Qwen3.5-0.8B
exr
99.0
3.3
74.5 ± 4.3
86.5
36.0
json
98.0
3.2
70.0 ± 2.8
85.7
37.8
exr-short
96.1
4.2
62.9 ± 6.9
82.8
35.4
tagged
95.9
2.1
73.0 ± 4.5
87.7
37.2
dsl
98.9
5.4
71.4 ± 4.8
80.4
37.8
Table 4: Guardrail at the full pool, percent, mean of 5 seeds. Neg. handled: held-out negatives refused (614 per run). Real refused: test orders refused ( Drefuse ). Corr.: 600 test orders with held-out correction cues; applied: the corrected attribute is right. Probes: hand-written corrections correct, of 40. Bold/underline as in Table 2 .
Figure 5: Exact match by training pool size, 95% bootstrap bands.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
1k
10k
100k
1M
2.5M
A
S
A
S
A
S
A
S
A
S
Qwen3.5-0.8B
exr
79.7 ± 4.9
17.0
86.0 ± 2.7
9.6
86.6 ± 2.0
9.2
86.6 ± 2.1
10.1
85.8 ± 1.6
10.3
json
70.4 ± 1.4
25.1
82.2 ± 1.3
14.6
83.0 ± 4.4
12.9
81.1 ± 4.6
11.5
81.6 ± 1.3
14.9
exr-short
73.5 ± 3.5
18.8
74.4 ± 4.9
15.0
76.2 ± 1.4
15.1
77.3 ± 2.9
13.4
75.2 ± 3.6
17.4
tagged
78.9 ± 3.7
16.8
84.6 ± 2.9
12.1
84.5 ± 2.5
10.8
84.1 ± 1.1
12.4
85.9 ± 2.9
11.6
Appendix
Table 5: Exact match A ( ± seed SD) and silent rate S at every pool size, percent.
Difference (points)
1k
10k
100k
1M
2.5M
Qwen3.5-0.8B
words − dsl
+6.7 [+3.4, +10.2] †
+4.8 [+2.3, +7.3] †
+6.2 [+3.9, +8.4] †
+5.8 [+3.4, +8.6] †
+7.8 [+6.1, +9.5] †
exr − json
+9.3 [+5.4, +12.8] †
+3.8 [+0.2, +6.7]
+3.6 [-1.1, +7.8]
+5.5 [+2.2, +9.2] †
+4.2 [+2.4, +5.8] †
exr − exr-short
+6.2 [+3.3, +9.2] †
+11.6 [+4.7, +16.6] †
+10.3 [+7.9, +12.7] †
+9.3 [+6.2, +12.5] †
+10.6 [+7.9, +13.4] †
words − exr
+4.0 [+0.5, +7.7]
+1.8 [-0.4, +4.4]
+1.6 [-0.8, +4.2]
+0.0 [-1.3, +1.3]
+1.8 [+0.1, +3.4]
S : exr − words
+11.4 [+8.9, +13.9] †
+5.1 [+3.3, +7.1] †
+4.4 [+2.8, +6.2] †
+5.0 [+3.1, +7.1] †
+5.8 [+4.3, +7.4] †
Appendix
Table 6: Table 3 with 95% hierarchical bootstrap intervals.
Format
Chars
Qwen3.5
LFM2.5
tok.
ch./tok.
tok.
ch./tok.
exr
123.0
45.0
2.7
58.0
2.1
json
150.7
45.6
3.3
64.7
2.3
exr-short
66.0
33.4
2.0
39.0
1.7
tagged
63.4
21.7
2.9
25.7
2.5
dsl
34.9
10.8
3.2
11.7
3.0
Appendix
Table 7: Mean characters and tokens of the gold targets over all 1,357 test orders, for both tokenizers.
Format
1e-4
2e-4
5e-4
Qwen3.5-0.8B
exr
84.77
85.82
80.36
json
83.62
83.62
80.27
edge
exr-short
76.82
79.89
79.98
edge
tagged
86.11
87.07
81.13
dsl
81.13
82.95
79.02
Appendix
Table 8: Learning-rate selection: dev exact match at a 100k pool, mean of 3 seeds; bold is the rate used. “edge”: chosen at the end of the grid.
Format
junk
long
non-order
off-menu
mixed
unknown word
probe
Qwen3.5-0.8B
exr
100.0
100.0
99.4
99.8
99.3
96.1
100.0
json
100.0
100.0
100.0
98.8
98.3
92.9
100.0
exr-short
99.7
100.0
99.4
97.5
96.3
87.0
100.0
tagged
100.0
100.0
99.1
95.9
96.9
86.3
100.0
dsl
100.0
100.0
99.4
99.8
99.0
96.0
100.0
Appendix
Table 9: Held-out negatives refused, by kind, full pool, percent.
Format
swap
retract
only
size
number
drink
cancel
restart
decoy
Qwen3.5-0.8B
exr
73.1
73.2
82.3
81.7
86.7
59.6
61.8
78.6
74.4
json
66.8
63.5
85.6
75.7
80.4
56.9
66.0
77.8
64.9
exr-short
60.3
58.0
79.7
72.6
74.9
46.3
57.6
61.0
59.6
tagged
72.3
67.2
84.7
75.2
82.7
63.8
67.6
79.2
70.1
dsl
72.9
72.1
80.3
72.1
78.9
62.8
67.8
75.6
63.3
Appendix
Table 10: Held-out corrections, exact match by kind, full pool, percent.
Format
n
line count
negation
topping swapped
value swapped
invented
dropped
other structure
Qwen3.5-0.8B
exr
698
17
19
15
31
9
9
1
json
1011
10
34
11
28
3
13
1
exr-short
1180
18
19
18
18
8
18
2
tagged
786
14
33
11
21
4
14
4
dsl
561
15
30
12
20
7
13
3
Appendix
Table 11: Silent errors by kind, full pool, all seeds pooled: n errors, and the percent of them in each kind.
Figure 6: Median latency at batch size one against output tokens, mean over five seeds; filled: Qwen, hollow: LFM2. Lines: least-squares fits.
Semantic parsing maps natural language into structured representations, such as logical forms, graphs, or queries, that machines can execute or reason over.