Context layers, curated documentation that an analytics agent fetches at query time, produce large accuracy gains on text-to-SQL benchmarks. A with/without comparison cannot say which part of the layer does the work: the semantic content, the retrieval scaffolding that delivers it, or the pre-computed views that usually accompany it. We report a four-arm ablation on DABStep on four models that separates the three. The instrument is a data contract: a YAML artifact that carries a domain's semantics and the rules an agent's tools enforce. One arm empties every field of prose in the frozen contract while holding the tool surface, retrieval instruction, table allow-list and operation rules byte-for-byte fixed. Compiling the contract's own SQL expressions into views gives the ceiling a pre-computed layer would reach: gold on all 176 tasks it covers. Content dominates. It raises hard-task accuracy from 13.9% to 55.1%, 22.6% to 56.6%, 22.9% to 68.4% and 37.0% to 77.4%, beating the same knowledge pasted into the prompt on every model. Scaffolding without content is worth 0 to 5 points on two flash models and 14 to 15 on two frontier models. Against the compiled ceiling the contract arm's shortfall is a failure to derive, and it falls from 39 points to 5 with model capability. The contract beats the prompt because the rule it needs is one lookup away rather than buried in a long prompt: its SQL carries the fee semantics up to 98% of the time against the prompt arm's 4%, and at the lowest cost per correct answer on three of four models. For practitioners: semantics first, scaffolding second, pre-computed macros only where an agent demonstrably fails to derive. Ungoverned arms submitted 166 mutating statements; governed arms none. The gain is confined to the contract's domain. On one model the benchmark's own withheld golds grade the contract arm at 51.9% against 18.8% for the prompt baseline.
Figures & tables
Views / macros
Declarative contract
Compiled
ahead of time, by a human
per query, by the model
Covers
anticipated metrics
any question in the domain
Cost
one macro per metric, maintained
one description per domain
Fails when
nobody anticipated the question
the model misapplies the rule
Table 1: Two ways to deliver the same semantics. The compiled contract of Section 5.3 is the left column built from the right column’s content, which is what lets this paper measure the difference between them.
no scaffolding
scaffolding
no content
schema
hollow
content
manual
contract
Table 2: The four arms as a two-factor design (artifact names: schema_only , contract_hollow , manual_prompt , contract ). hollow differs from contract only in that its prose fields are empty. manual carries the same knowledge as contract in its original form, pasted into the prompt.
Model
Pinned id
Route
T
Tasks
GLM
glm-5.3-flash
OpenRouter, fp8
0
401
DeepSeek
deepseek-v4-flash
OpenRouter, fp8
0
279
Sonnet 5
Claude Sonnet 5
Bedrock via gateway
—
401
GPT-5.6
gpt-5.6-sol
OpenRouter
—
401
Table 3: The four models, in the order used throughout: bare-schema hard accuracy. This is not the order they were run in; Sonnet 5 ran last. Each is pinned to one provider endpoint with fallbacks disabled. T is temperature; a dash means the route does not accept the parameter. DeepSeek’s task count is its complete-case set, defined below.
Arm
Overall
Hard
Easy
Cost
GLM ( glm-5.3-flash , 401 tasks)
schema
22.7%
13.9%
65.2%
$0.76
hollow
26.4%
19.3%
60.9%
$0.85
manual
31.7%
22.9%
73.9%
$0.96
contract
57.9%
55.1%
71.0%
$0.67
DeepSeek ( deepseek-v4-flash , 279 tasks)
Table 4: Accuracy and cost, models in order of bare-schema capability. Hard splits are n=332 , 226, 332 and 332; easy are 69, 53, 69 and 69. contract is the most accurate arm on all four models, the cheapest on the two flash models, and within about 3% of the cheapest on Sonnet 5. Arms are in ladder order, so the two content-free arms are adjacent.
Comparison
GLM
DeepSeek
Sonnet 5
GPT-5.6
schema vs contract
3×10−31 14/ 155
2×10−15 14/ 93
1×10−35 11/ 162
5×10−32 12/ 152
manual vs contract
2×10−17 29/ 134
0.0067 30/ 56
5×10−27 22/ 160
2×10−17 12/ 96
hollow vs contract
9×10−24 23/ 149
8×10−17 13/ 96
2×10−17 25/ 126
2×10−17 18/ 111
schema vs manual
7×10−6 14/50
2×10−12 5/58
0.066 15/28
2×10−8 22/78
manual vs hollow
0.033 55/34
4×10−13 63/6
1×10−4 27/ 64
0.44 58/49
schema vs hollow
0.058 20/35
0.61 19/15
9×10−9 14/ 64
4×10−7 20/ 67
Table 5: Every pairwise paired McNemar over all tasks: p -value, then discordant pairs won by the left arm / by the right arm. The last row is the scaffolding step of the decomposition, on which the flash and frontier models split two and two. Sonnet 5’s fourth row is its own result: the pasted manual is not distinguishable from a bare schema.
Figure 2: The two steps of the decomposition on hard tasks, four models in order of bare-schema capability. Adding the nine governed tools, the retrieval instruction, the table allow-list and the operation rules with the prose emptied ( schema to hollow ) moves accuracy by +5.4 , +0.0 , +15.1 and +14.2 points; restoring the prose ( hollow to contract ) moves it by a further +35.8 , +34.0 , +30.4 and +26.2 . Bars are Wilson 95% intervals.
GLM
DeepSeek
Sonnet 5
GPT-5.6
Hard tasks
macro
derived
macro
derived
macro
derived
macro
derived
schema
2.8%
25.0%
8.5%
34.7%
6.8%
37.8%
31.8%
41.2%
hollow
10.2%
25.7%
10.2%
31.7%
36.4%
37.2%
46.6%
54.1%
manual
14.8%
32.4%
38.1%
44.6%
8.5%
38.5%
49.4%
48.6%
contract
60.8%
45.9%
62.7%
46.5%
73.9%
60.1%
94.9%
56.1%
compiled contract
100%
—
100%
—
100%
—
100%
—
Table 6: Hard tasks split by whether the compiled contract answers the task outright ( macro , n=176 ; 118 on DeepSeek’s complete-case set) or requires reasoning on top of it ( derived , n=148 ; 101). The gap between contract and the compiled contract on macro is derivation failure on information the agent provably had.
Wrote all six clauses
GLM
DeepSeek
Sonnet 5
GPT-5.6
schema
7%
4%
3%
8%
hollow
14%
4%
4%
8%
manual
3%
9%
4%
4%
contract
39%
65%
55%
98%
Table 7: Share of the 97 tasks requiring all six fee clauses (69 on DeepSeek) on which the agent’s submitted SQL expresses all six. manual holds the same knowledge as prose in its system prompt. contract against manual is p=2×10−10 , 3×10−12 , 9×10−16 and 2×10−47 (Fisher exact).
schema
hollow
manual
contract
GLM
Turns / task
12.1
8.4
9.8
7.1
Tool calls
5,134
4,110
4,282
3,355
Reasoning tokens
428k
525k
400k
154k
Cost / correct
$0.0084
$0.0080
$0.0076
$0.0029
Forced answers
41
0
18
0
Table 8: Work and cost per arm. The contract arm uses the fewest turns and the least reasoning on every model and never exhausts its output budget; it is the cheapest arm per correct answer on three of the four. DeepSeek’s reasoning row is per task rather than a total, because its complete-case set has fewer rows than the other runs.
GPT-5.6
Fresh input
Cached input
Output
Total
manual
$3.55
$3.85
$7.02
$14.42
contract
$8.87
$3.78
$8.67
$21.32
Table 9: GPT-5.6’s bill for the two content-bearing arms, decomposed from the recorded token counts at the pinned endpoint’s prices: 2.00permillionfreshinputtokens,0.20 cached, $10.00 output (reasoning included).
schema
hollow
manual
contract
GLM
27 (8)
0
92 (13)
0
DeepSeek
24 (2)
0
1 (1)
0
Sonnet 5
0
0
22 (2)
0
GPT-5.6
0
0
0
0
Total
51 (10)
0
115 (16)
0
Table 10: Mutating statements submitted, by arm (tasks in parentheses). Fisher exact on traces containing at least one: p=8×10−7 for GLM alone, p=3×10−8 pooled over the three models where any arm attempted a write. DeepSeek and Sonnet 5 alone are underpowered ( p=0.25 and 0.50 ).
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
GLM
DeepSeek
Sonnet 5
GPT-5.6
Arm
exp.
corr.
exp.
corr.
exp.
corr.
exp.
corr.
contract
15.8
50.1
17.2
51.0
10.6
64.2
10.5
72.7
hollow
26.6
16.3
21.6
19.8
14.3
34.8
14.7
46.9
manual
19.7
20.3
23.7
37.1
40.5
18.2
13.8
46.3
schema
37.0
10.9
31.4
18.5
36.8
18.0
10.6
34.8
Appendix
Table 11: Exposure to scorer leniency: the share of each arm’s correct hard-split answers that required more than exact match against the reconstructed gold (exp.), and the accuracy that follows from correcting it (corr.). An upper bound.
Model
Bucket
schema
hollow
manual
contract
GLM
fee (294)
13.9%
18.7%
24.5%
59.2%
GLM
non-fee (38)
5
9
4
9
DeepSeek
fee (196)
23.0%
22.4%
44.9%
61.2%
DeepSeek
non-fee (30)
6
7
9
8
Sonnet 5
fee (294)
22.4%
40.1%
24.1%
73.5%
Sonnet 5
non-fee (38)
10
8
8
11
Appendix
Table 12: Hard-task accuracy split by whether the question concerns the domain the contract describes. Non-fee cells are counts, not rates.
Figure 3: Hard accuracy against base-model capability, where capability is the bare-schema arm’s own accuracy. The margin over manual is +32.2 , +13.7 , +44.6 and +27.1 : not monotone.