One common trade-off in the use of large language models involves reducing the size of the model while increasing the amount of computation at inference time, for example by using a wider beam search. In this paper, we examine the constrained case of this "model size vs. inference compute" trade-off, in which the model outputs are constrained by a strict grammar at inference time. Our results demonstrate that the constrained trade-off behaves differently from the unconstrained trade-off. We investigate the task of converting a prose query into an equivalent SQL query (text-to-SQL). Performance is evaluated on the Spider text-to-SQL benchmark, using the Qwen2.5-Instruct model family ranging in size from 0.5B to 7B parameters, all at 4-bit precision. We experiment with two approaches to varying inference compute: (i) beam search with a variable number of beams; and (ii) sample+vote, i.e., sampling several constrained outputs and then voting on their execution results, where the number of samples is varied. On the 1034-example development set, we find that: (a) both beam search and sample+vote improve accuracy, especially on smaller model sizes; (b) the "model size vs.\ inference compute" trade-off is not advantageous in this experiment, because moving to a larger model size typically results in higher accuracy than increasing inference compute on the same model size; (c) beam search outperforms sample+vote at a matched inference budget. This latter result is of particular interest since it contrasts with the findings of the unconstrained trade-off.
Figures & tables
Model
No-grammar greedy
Method
Budget 1
Budget 2
Budget 4
Budget 8
Qwen2.5-0.5B
0.135
Beam
0.139
0.203
0.238
0.245
Sample+vote
0.113
0.159
0.201
0.248
Qwen2.5-1.5B
0.326
Beam
0.351
0.482
0.502
0.505
Sample+vote
0.299
0.348
0.403
0.450
Qwen2.5-3B
0.477
Beam
0.445
0.516
0.549
0.562
Sample+vote
0.441
0.469
0.489
0.513
Table 1: Execution accuracy on the Spider development set ( n=1034 ) by model size, method, and budget. The no-grammar greedy baseline is shown for reference. Each cell is a single decoding run over the full set.
Figure 1: Execution accuracy versus beam width for each model size. The benefit of wider beams decreases with size and saturates by width 4. Error bars are 95% Wilson score intervals over the n=1034 test examples, and each point is one decoding run (beam search is deterministic).
Figure 2: Beam search compared with sample+vote at matched budget. Sample+vote is never significantly more accurate than beam search. Beam search is significantly ahead in 11 of the 16 cells by an exact McNemar test on the paired predictions (see Table 2 ) and significantly behind in none; the remaining five cells are statistical ties. Error bars are 95% Wilson score intervals for the individual accuracies. As discussed in Section 4 , each point represents a single decoding run (employing a single random number seed for sample+vote).
Accuracy
Correct examples
Model
B
Beam
Sampling
Beam only
Sampling only
Agree
p
Qwen2.5-0.5B
1
0.139
0.113
73
46
915
0.017∗
2
0.203
0.159
103
57
874
3.4×10−4∗
4
0.238
0.201
115
77
842
0.0074∗
8
0.245
0.248
95
98
841
0.89
Qwen2.5-1.5B
1
0.351
0.299
116
62
856
6.3×10−5∗
Table 2: Paired comparison of beam search and sample+vote. The table shows the significance of the difference between each pair of points in Figure 2 . Because beam search and sample+vote are both evaluated on the same n=1034 examples, the appropriate test is McNemar’s exact test on the discordant pairs rather than an interval for two independent proportions. “Beam only” counts examples beam answered correctly and sampling did not, and conversely for “Sampling only”; “Agree” counts examples on which the two methods gave the same verdict. Asterisks mark p<0.05 .
Inference-time scaling studies how extra computation during deployment can improve model performance through search, sampling, verification, or adaptive reasoning.
Large Language Models (LLMs) have recently improved Text-to-SQL generation, but their computational cost can hinder deployment in resource-constrained settings. This paper evaluates the Text-to-SQL capabilities of small and medium open-source LLMs (1B-8B parameters) under a controlled inference budget, focusing on how model specialization and prompt formulation affect performance. Using the Spider benchmark, we compare multiple prompt representations and assess the accuracy of generated queries. Our results show that SQL-specialized models are consistently more reliable than general-purpose and chat-oriented models at small scales, and that performance can vary substantially depending on the prompt used. To reduce this variability without resorting to larger models, we introduce three lightweight inference workflows that automatically retry with alternative prompts and/or models when failures occur, including an optional syntax-correction step. These workflows significantly improve end-to-end success rates, with a trade-off in additional inference calls. These results demonstrate a practical path to stronger Text-to-SQL performance with small LLMs, by trading modest additional inference cost for higher reliability.
While recent advancements in inference-time learning have improved LLM reasoning on Text-to-SQL tasks, current solutions still struggle to perform well on the most challenging tasks in the Bird-Bench (BIRD) benchmark. This is due to inadequate solution space exploration, which is necessary to uncover promising candidate queries that can be further refined to produce the correct output. To address this challenge, we introduce CA-SQL, a novel Text-to-SQL pipeline that utilizes the estimated difficulty of a task to dynamically scale the breadth of the exploration for generating solution candidates. In addition, we use a custom prompt seeding method, based on principles of evolutionary search, to further elicit exploratory behavior from the base LLM and a novel voting method to select the best candidate solution at the end of the search. Experiments demonstrate that our solution achieves a state-of-the-art score of 51.72% on the "challenging" tier of BIRD development set problems, using only GPT-4o-mini, out-performing other in-context learning approaches, even those that leverage larger models. Overall, our method attains a competitive 61.06% execution accuracy and 68.77% Soft F1 score on the BIRD development dataset.
Large language models have achieved remarkable progress on Text-to-SQL through reasoning-enhanced fine-tuning, yet existing approaches predominantly rely on massive instruction corpora under the assumption that scale drives performance. We challenge this paradigm by investigating a fundamental question: what is the minimal data requirement for effective Text-to-SQL instruction tuning? We propose LIMIT(Less Is More for Instruction Tuning in Text-to-SQL), a data-centric framework that demonstrates strong database reasoning can emerge from an extremely compact training set when examples are strategically selected. LIMIT operates through four stages: difficulty-aware filtering that identifies samples within the model's learning frontier, chain-of-thought synthesis with consistency-based selection, multi-dimensional quality scoring via LLM-as-judge, and genetic algorithm optimization that jointly maximizes schema coverage and sample quality. On the BIRD and Spider benchmark, LIMIT selects only 796 and 863 samples while achieving 100% table coverage, enabling Qwen3-8B to reach 69.1% and 88.9% execution accuracy.This result surpasses methods trained on 20 times more data and establishes a new state-of-the-art among open-source approaches. Our findings suggest that careful data curation, rather than scale, is the key to efficient Text-to-SQL learning.