LYNX: Analytical Program Synthesis via Grounding and Closed-Loop Refinement
Authors: Ruichen Xia, Xiao Han, Dingyu Yang, Luwei Yang, Sai Wu, Dongxiang Zhang, Binhui Yao, Gang Chen
Organizations: The State Key Laboratory of Blockchain and Data Security, Zhejiang University, Hangzhou, China · Institute of Computing Innovation, Zhejiang University, Hangzhou, China · Zhejiang Geely Holding Group Co., Ltd., Hangzhou, China
LLM-powered data analytics aims to translate natural language instructions into executable analytical programs over underlying data. However, existing approaches, whether treating LLMs as intent-to-code translators or as standalone analytical engines, often fail to reliably align domain semantics with complex schemas and heterogeneous metadata, resulting in brittle execution and semantically inconsistent analytical outputs in real-world workloads. We present LYNX, a multi-agent framework that formulates LLM-driven analytics as a schema- and knowledge-constrained analysis problem rather than a single-pass generation task. LYNX organizes analytical synthesis into a closed-loop workflow of planning, generation, verification, and refinement through three role-specialized agents that collaboratively enforce schema grounding, semantic consistency, and execution correctness. To address domain ambiguity and schema misalignment, LYNX integrates Semantic-Structural Distillation and Stage-Aware Allocation (SSD-SAA). A delta-guided iterative revision continuously diagnoses and repairs semantic and execution-level failures, substantially improving synthesis robustness. We evaluate LYNX on two NL-to-SQL benchmarks, BIRD-SQL Mini-Dev and the more challenging Spider 2.0-Lite. Experimental results demonstrate strong performance and efficiency, achieving 10.66% to 59.54% execution accuracy across lightweight and commercial models, while reducing token consumption by 70.53% compared with the prior most efficient method. Our framework displays high generalization over different backbones, from open-source, lightweight models to the latest commercial models. Additional case studies on quantitative finance analytics and complex multi-relational SQL workloads further demonstrate the framework's robustness, scalability, and practical applicability in real-world analytical scenarios.
Figures & tables
Fig. 1 : The Framework of the Multi-agent System
Fig. 2 : Decoupled Domain Knowledge Example
Fig. 3 : Risks Contents Fallback Mechanism
Fig. 4 : Rubber Duck Debugging Strategy
Model
Method
Easy
Medium
Hard
Total EX
Avg. Tokens
GPT-4o
DIN-SQL
5.79
0.43
0.00
1.46
—
CHESS
9.92
3.00
1.24
3.84
—
DAIL-SQL
13.20
5.58
1.24
5.68
—
LYNX (ours)
44.36
37.25
22.14
34.05
13.8K
DeepSeek-R1
RSL-SQL
—
—
—
30.53
50.0K
LinkAlign
—
—
—
33.09
—
TABLE I : Comparison of performance results on Spider 2.0-Lite
Backbone Model
EX %
Avg. Times
Avg. Tokens
Qwen3-14B
10.66
6.50
48.4K
GPT-4o
34.05
5.16
13.8K
DeepSeek-V3
35.45
4.25
13.4K
DeepSeek-R1
51.13
4.80
11.2K
Claude-Sonnet-4.6
57.58
4.29
7.1K
Gemini-3.1-Pro
59.54
3.70
6.4K
TABLE II: LYNX performance on different backbones
Metric
DeepSeek-V3
Qwen3-14B
Executability
88.42%
49.36%
Avg. Attempts
4.25
6.50
Few-Shot ( ≤ 3 attempts)
68%
43%
Executable Rate
TABLE III : Comparison of Executability Performances
Fig. 5 : Attempt Distribution of Models
Methods (Models)
EX (%)
VES
Soft-F1
TA + codellama-34B [ 37 ]
27.57
-
-
GPT-3.5-turbo [ 49 ]
38.00
37.33
41.84
llama3-70B-instruct
40.80
39.02
44.38
Claude-2 [ 50 ]
42.70
-
-
TA + llama3-70B-instruct [ 48 ]
42.80
41.37
46.66
TA + DeepSeek [ 48 ]
43.74
-
-
TABLE IV : Performance Comparison on BIRD Mini-Dev
Model
Metric
Simple
Moderate
Hard
Total
Qwen3-14B
EX
57.43
39.20
33.33
43.40
VES
54.11
36.10
31.44
40.48
Soft-F1
62.95
45.22
42.55
49.92
DeepSeek-V3
EX
63.51
47.20
34.31
49.40
VES
58.69
43.02
32.14
45.44
Soft-F1
68.22
54.91
42.50
56.31
TABLE V : Detailed Results across Difficulty Levels
Fig. 6 : Scores on Different SQL Dialects
agent
function
EX % ( Δ )
-
Full model (Ours)
35.45
Generator
w/o SSD-SAA
≈0 ( ↓≈ 100% )
Critic
w/o Delta-guided memory
27.36 ( ↓ 22.8% )
Reviser
w/o Iterative revision
22.43 ( ↓ 36.7% )
w/o Logical verification
30.23 ( ↓ 14.7% )
TABLE VI: Ablation Results on Spider 2.0-lite
Agent
Tokens
Runtime
Prompt
Completion
Generator
3520±627
496±102
14.94±4.08s
Critic
3462±1128
934±228
6.82±1.40s
Reviser
2438±413
473±113
10.16±3.92s
TABLE VII: Token Consumption and Runtime Analysis
Fig. 7 : Comparison of History Length after Attempts
Fig. 8 : Details for Stage-aware Reasoning Efficacy
TABLE VIII : Revision Loop Effects across Models
Metric
Method
Difficulty
Simple
Moderate
Hard
Total
EX
w/o Semantics
62.16
44.00
36.27
47.80
w/ Semantics
63.51
47.20
34.31
49.40
Δ
+1.35
+3.20
-1.96
+1.60
VES
w/o Semantics
56.74
40.17
34.23
43.86
w/ Semantics
58.69
43.02
32.14
45.44
TABLE IX : Semantic Impact to Subsets on BIRD
Fig. 9 : Definitions for External Knowledge
Fig. 10 : The Pipeline of the Generated Python Script
Figure 20
Fig. 13 : Case Multi-step Process
Stage
general-purpose LLM (GPT-5.2)
Our Framework (w/ revision)
Schema
Hallucinate table name
use correct FQN
Challenge
Standard casting fails on dirty data
Detected error and applied SAFE_CAST
Logic
Implicit type mismatch in comparisons
Explicitly compare INT64 vs STRING
Outcome
Execution Error "404 Not Found" ✗
Successfully executed Correct Answer ✓
TABLE X : Comparative Analysis of Case "bq282"
Fig. 14 : An Example of Challenges
Fig. 15 : Stage-aware Reasoning
Date
Open
High
Low
Close
…
Industry
2009-01-05
55.09
56.84
54.76
56.73
…
Real Estate
2009-01-06
56.73
59.03
56.18
59.03
…
Real Estate
⋮
⋮
⋮
⋮
⋮
⋱
⋮
2025-08-15
285.33
294.87
283.35
291.29
…
Real Estate
TABLE XI : Sample of Stock Trading Data (000006.SZ)
Large language model (LLM)-based program synthesis turns natural-language goals and examples into executable programs, often using tests or formal constraints.