Much of the information needed for data analysis is buried in text columns: an analyst may infer customer frustration from complaint text and use it to explain resolution time. We call workloads in which processing text columns is central to subsequent data analysis TADA (Text Analysis in Data Analytics). In an agentic setting, the user's query dynamically specifies this processing, and answering it interleaves LLM-based semantic steps with exact SQL or Python operations. This mixed execution raises two design questions: which work should be neural and which symbolic, and which semantic outputs later steps can reuse. We address them for agent-planned TADA workflows with two complementary, query-driven semantic atoms. Each atom takes a sequence of text items and a natural-language query specifying the semantic task: Tagging returns one string per input item, while Categorization returns M category objects over N input items. Tagging results are materialized as a row-aligned column and Categorization results as a category table; SQL or Python then performs exact computation over this exposed state. We realize this interface in TADA-Agent, whose atoms run inside generated Python programs and internally handle inputs beyond a reliable context window. We evaluate TADA-Agent on TABench, a benchmark of 40 complete analysis requests over spreadsheet tables spanning workflow composition, input--output structure, and literature-grounded analytical scenarios. Against a semantic-operator baseline and relative to a held-out TADA reference run, TADA-Agent delivers comparable row-level semantic quality, has higher complete-artifact preservation estimates, and uses over an order of magnitude fewer semantic-model requests under the evaluated native execution paths.
Figures & tables
Representative request
Scenario
Input–output
TADA workflow
Existing operator
Workflows corresponding to familiar semantic-operator families
Label each review as positive, negative, or neutral
Classification
N×1→N×1
Tag with the three labels written in the query
sem_map
Discover issue categories and group the reviews by them
Grouping; classification
N×1→(M,N×1)
Categorize → build a query from the category names → Tag → GroupBy
Semantic grouping
Keep comments that imply a risk of customer churn
Semantic filtering
N×1→N′×1
Tag a Yes / No decision → Filter
sem_filter
Extract time, location, scope, and cause from each incident
Information extraction
N×1→N×w
Invoke Tagging w times, once for each output column
sem_map / sem_extract
Produce one overall summary of the survey responses
Text summarization
N×1→1
Categorize with a query requesting one overall summary
sem_agg
Table 1: Representative TADA requests viewed by scenario, input–output structure, workflow composition, and correspondence to existing semantic operators. Scenario names follow the fine-grained intent taxonomy; the appendix groups these leaves into 12 broader reporting classes.
Scenario class
TADA-Agent
Semantic operators
Claude Code
Code-only
Direct
Classification
98.6 [97.0, 100.0]
97.5 [96.1, 98.8]
97.9 [95.2, 100.0]
96.6 [94.0, 98.8]
94.7 [91.3, 98.0]
Field extraction
98.9 [96.6, 99.9]
98.2 [95.5, 99.7]
97.9 [93.9, 99.7]
90.6 [80.3, 98.4]
82.5 [72.2, 95.0]
Semantic ranking (Spearman ρ )
0.801 [0.600, 0.980]
0.752 [0.483, 0.957]
0.774 [0.558, 0.962]
0.590 [0.249, 0.921]
—
Constrained generation
89.1 [83.8, 94.0]
78.6 [64.2, 88.8]
84.2 [74.8, 90.6]
35.0 [27.5, 42.8]
60.3 [55.2, 64.0]
Table 2: Row-level semantic quality on TABench by scenario class. Except for the semantic-ranking row, each entry is the percentage of evaluated units judged at least as good as the common reference output, with a 95% case–run cluster-bootstrap interval over three independent runs per configuration (identical inputs, models, and judge). The semantic-ranking row reports mean within-case Spearman ρ against the same reference, macro-averaged over the four ranking cases and three runs, with a case–run cluster-bootstrap 95% interval computed the same way (wide because only four cases enter this row); the complementary absolute-score agreement is reported in the text, and Direct is omitted because its incomplete/malformed Q34 artifacts yield method-dependent row coverage. Bold marks the highest point estimate and does not imply statistical significance; classes are never aggregated.
Scenario class
TADA-Agent
Semantic operators
Claude Code
Code-only
Direct
Grouping and clustering
8/9 [56.5, 98.0]
5/9 [26.7, 81.1]
8/9 [56.5, 98.0]
6/9 [35.4, 87.9]
5/9 [26.7, 81.1]
Semantic filtering
9/12 [46.8, 91.1]
8/12 [39.1, 86.2]
6/12 [25.4, 74.6]
4/12 [13.8, 60.9]
0/12 [0.0, 24.2]
Aggregation and summarization
12/12 [75.8, 100.0]
6/12 [25.4, 74.6]
12/12 [75.8, 100.0]
11/12 [64.6, 98.5]
12/12 [75.8, 100.0]
Numerical and temporal analysis
6/9 [35.4, 87.9]
4/9 [18.9, 73.3]
6/9 [35.4, 87.9]
6/9 [35.4, 87.9]
1/9 [2.0, 43.5]
Anomaly detection
6/6 [61.0, 100.0]
4/6 [30.0, 90.3]
3/6 [18.8, 81.2]
5/6 [43.6, 97.0]
6/6 [61.0, 100.0]
Table 3: Complete-artifact preservation on TABench. Each entry counts outputs judged at least as complete and correct as the fixed reference, with descriptive 95% Wilson intervals in percent. The unit is a delivered artifact rather than a row; functionally determined identity columns do not count as missing content. These results are specific to this reference output and are not combined with Table 2 .
Metric
TADA-Agent
Sem. operators
Task-runs
120
120
Successful requests
451
≥ 18,852
Mean calls per task-run
3.76
≥ 157.10
Relative count
1.0 ×
≥ 41.8 ×
Table 4: Recorded calls to the dedicated semantic-processing LLM across three runs of each TABench task. “Mean per task-run” is the mean number of such LLM calls, not elapsed time. The ≥ symbol marks conservatively incomplete request logs.
Workload
System
Accuracy
Macro-F1
Acc./agreement
Sentiment 32,179 rows
TADA-Agent
0.973±0.000
0.928
1.017
Claude Code
0.817±0.047
0.577
0.854
LOTUS †
0.954
0.884
0.997
Topic assign. 32,179 rows
TADA-Agent
0.942±0.002
0.838
1.038
Claude Code
0.856±0.116
0.729
0.943
LOTUS †
0.881
0.838
0.970
Table 5: Large-scale end-to-end execution. Accuracy is agreement with the majority-vote labels. Acc./agreement divides accuracy by inter-annotator agreement (1.0 means they match). TADA-Agent and Claude Code report mean ± s.d. over three full-scale runs; LOTUS rows are measured subsets: 500 rows for the 32K workloads and the first 2,050 rows for the 6.6K workload ( † ). Macro-F1 and Acc./agreement are also means across valid full-scale repetitions; only Accuracy displays its across-run s.d.
Figure 3: Probability of recovering the true global top-5 categories over 5,000 random partitions of each fixed 32K-row population as the local retention budget varies from r=5 to 20. The dashed line is the 0.95 target; outlined circles mark pilot-selected values. The blue separated distribution needs no extra candidates; the five orange curves progressively narrow the cutoff gap.
Figure 4: Public document-extraction benchmarks (higher is better). (a) Reproduced CUAD average F1; TADA-Agent, DocETL, and LOTUS rerun with the same GPT-5.5 model and evaluator. (b) Reproduced BioDEX report precision at 5 and 10 predictions (RP@5/10) and recall of gold adverse-event terms under the same model, candidate budget, and official evaluator; bars are means over completed runs, and error bars show the observed min–max where repeats exist. Provenance caveats are stated in the text.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Atom
Input
Raw return
Materialization
Large-scale path
Tagging
Text sequence X and atom-level query q
One string si for each xi , preserving length and order
One row-aligned column associated with input rid values
Independent: partition and reassemble; joint: discover a shared category list, then use the independent path
Categorization
Text sequence X and atom-level query q
Category objects with a required name and optional description
One category table with fresh cid values
Partition, retain local candidates, semantically consolidate, and select the requested global list
Appendix
Table 6: Public atom contracts and their materialized analytical state.
N
Monolithic sort
Monolithic filter F1
Monolithic agg.
Atom+symbolic
100
79/100
1.000
No
Exact
200
161/200
0.765
No
Exact
500
280/500
0.678
No
Exact
1000
400/1000
0.105
No
Exact
Appendix
Table 7: Single-run neural–symbolic boundary diagnostic. Sort coverage reports returned identifiers over requested rows; aggregation reports exact agreement on count, sum, and average.
ID
Scenario class
Cases
Dominant program structure
S1
Row-wise classification
Q01–Q03, Q27, Q28
Tag typed labels against a fixed or inferred label set (one string value per row per call).
S2
Open grouping & clustering
Q04, Q05, Q36
Categorize a corpus-level taxonomy, optionally Tag rows with its identifiers; variants add outlier isolation.
S3
Semantic filtering
Q06–Q08, Q29, Q38
Tag a predicate per row, then deterministic row selection; includes negated, implicit, and alias-sensitive predicates.
S4
Field extraction
Q09, Q10, Q30, Q39, Q40
Tag typed fields per row from free text; repeated per output column; includes open-set and relational attributions.
S6
Aggregation & summarization
Q13–Q15, Q31
Group rows deterministically, Categorize or summarize per group, compare across groups or time.
S7
Numerical/temporal analysis
Q16, Q17, Q32, Q42
Combine semantic state with numeric filtering, aggregation, or multi-column fusion.
Appendix
Table 8: TABench task inventory by scenario class. Dominant program structure describes the typical composition; a query may exercise more than one class’s mechanics. S5 and S8 are boundary classes outside the two-atom contract and are discussed rather than scored.
Units
Maj.
Agree
Flip
By stratum
Contested
21
19
17
1
Low-confidence
12
8
4
1
High-confidence
6
6
5
0
By task type
Summarization (Q24)
8
8
8
0
Appendix
Table 9: Human calibration of the LLM judge on 39 stratified judgment units, six annotators. Maj. = units with a human majority verdict; Agree = judge matches human majority; Flip = directional conflict (one side A, the other B).
Workload
Rep
Calls
Prompt
Compl.
Cache
Wall
TADA-Agent (provider-reported)
Sentiment
1
33 (105)
–
–
–
164.6
Sentiment
2
33
891.7
349.2
–
15.1
Sentiment
3
33
891.7
259.9
–
12.2
Topic
1
33
894.7
623.5
–
29.2
Topic
2
33
894.7
674.5
–
31.8
Appendix
Table 10: Per-repetition execution cost. Calls: planned LLM calls for TADA-Agent (observed attempts in parentheses where they differ), API messages for Claude Code, per-row calls for LOTUS. Tokens in thousands; “cache” is Claude Code prompt-cache reads. Wall time in minutes. ∗ : transcript token fields incomplete; –: not recorded or not aggregable.
Workload / rep
Discovered
Unmapped
Mapped acc.
Topic 32K / 1
10
4
0.690
Topic 32K / 2
10
3
0.675
Topic 32K / 3
10
7
0.685
Category 6.6K / 1
12
8
0.296
Category 6.6K / 2
11
6
0.134
Category 6.6K / 3
10
4
0.195
Appendix
Table 11: Open-vocabulary repetitions: discovered vocabulary size, labels with no reference correspondence, and mapped accuracy.
Regime
p5−p6
Oracle r⋆
r
Q.05
Rec. at r=5
Rec. at r
Separated
0.061891
5
5
1.00
1.000
1.000
Narrow 1
0.002293
9
10
0.90
0.909
1.000
Narrow 2
0.001963
9
10
0.95
0.871
1.000
Narrow 3
0.001634
9
10
0.95
0.824
1.000
Narrow 4
0.001306
10
10
0.95
0.744
0.990
Narrow 5
0.001175
10
10
0.95
0.739
0.985
Appendix
Table 12: Planner position relative to the full r=5,…,20 recovery matrix. The oracle is the first r whose recovery and every larger tested r remain at least 0.95. Empirical recovery uses 5,000 partitions of each fixed population.
Large language models (LLMs) have enabled data-science agents to automate multi-step analyses over heterogeneous files. However, incorrect choices regarding data sources, scope, or statistical definitions often lead to silent errors: computations execute successfully but produce plausible yet incorrect outputs that fail to answer the intended question. To mitigate this, we present RADAR, an auditable runtime that makes an agent's analytical choices inspectable and supports their revision through execution feedback. RADAR operates through three core mechanisms. First, an evidence-preserving exploration module retrieves task-relevant content while retaining source locations and observation coverage. Next, the runtime uses typed operators to record the agent's declared inputs, operation arguments, and resulting observations. Finally, runtime validation checks proposed operations against these observations. When a conflict is detected, the runtime rejects the operation or provides diagnostic feedback, allowing the agent to revise its choices before errors propagate. This design enables agents to fail loudly while leaving semantic interpretation to the LLM. On KramaBench, RADAR achieves overall scores of 0.723 with full source retrieval and 0.747 with gold sources supplied, corresponding to relative gains of 35.9% and 28.8% over the strongest baselines. Beyond KramaBench, RADAR achieves relative performance gains of 14.0% on DA-Code and 59.3% on DABStep, demonstrating its applicability across diverse agentic data-analysis workflows.
Context layers, curated documentation that an analytics agent fetches at query time, produce large accuracy gains on text-to-SQL benchmarks. A with/without comparison cannot say which part of the layer does the work: the semantic content, the retrieval scaffolding that delivers it, or the pre-computed views that usually accompany it. We report a four-arm ablation on DABStep on four models that separates the three. The instrument is a data contract: a YAML artifact that carries a domain's semantics and the rules an agent's tools enforce. One arm empties every field of prose in the frozen contract while holding the tool surface, retrieval instruction, table allow-list and operation rules byte-for-byte fixed. Compiling the contract's own SQL expressions into views gives the ceiling a pre-computed layer would reach: gold on all 176 tasks it covers. Content dominates. It raises hard-task accuracy from 13.9% to 55.1%, 22.6% to 56.6%, 22.9% to 68.4% and 37.0% to 77.4%, beating the same knowledge pasted into the prompt on every model. Scaffolding without content is worth 0 to 5 points on two flash models and 14 to 15 on two frontier models. Against the compiled ceiling the contract arm's shortfall is a failure to derive, and it falls from 39 points to 5 with model capability. The contract beats the prompt because the rule it needs is one lookup away rather than buried in a long prompt: its SQL carries the fee semantics up to 98% of the time against the prompt arm's 4%, and at the lowest cost per correct answer on three of four models. For practitioners: semantics first, scaffolding second, pre-computed macros only where an agent demonstrably fails to derive. Ungoverned arms submitted 166 mutating statements; governed arms none. The gain is confined to the contract's domain. On one model the benchmark's own withheld golds grade the contract arm at 51.9% against 18.8% for the prompt baseline.
LLM-based agents for text-to-SQL often struggle with latency-performance trade-off, where performance improvements come at the cost of latency or vice versa. We reformulate text-to-SQL generation within the lens of software test coverage where the original query is prepared with a suite of test cases with simpler, atomic SQLs that are executed in parallel and together ensure semantic coverage of the original query. After iterating on test case coverage, the final SQL is generated only when enough information is gathered, leveraging the explored test case SQLs to ground the final generation. We validated our framework on a state-of-the-art benchmark for text-to-SQL, Spider 2.0, achieving a new state-of-the-art with 70.2% execution accuracy.