Recent advances in large language models (LLMs) have established human-agent collaboration as a promising paradigm for data science. Effective collaboration requires a shared computational abstraction that enables agents to reason about processing logic while allowing human analysts to understand, guide, and refine the analysis. Although scripts written in Python or SQL provide precise control, their syntax and execution details can be difficult for analysts to understand and modify. In this paper, we argue that dataflows are an effective abstraction for human-agent collaboration in data science. We introduce BobFlow, a system that combines the dataflow abstraction with the ReAct paradigm. It enables agents to iteratively construct and execute dataflows for data science tasks while allowing analysts to inspect the analysis and provide feedback. Our user study shows that BobFlow helps analysts understand, inspect, and verify agent-generated analyses. A thorough experimental evaluation across proprietary and open-weight LLMs shows accuracy score gains of up to 18.6 points at just over 1/3 of a script-based ReAct agent's inference cost.
Figures & tables
Figure 1. Three abstractions of the same movie-analysis task. Natural language leaves decisions unspecified; Python is precise but harder to understand and modify; dataflows expose operators, dependencies, and intermediate results for targeted feedback.
Figure 2. Human–agent collaboration on data science tasks.
Abstraction
Precise [-2pt] Semantics
Intuitive [-2pt] Representation
Step-Level [-2pt] Transparency
Natural Language
✗
✓
✗
Programming Language
✓
✗
✓
SQL
✓
✗
✗
Dataflow
✓
✓
✓
Table 1. Abstractions for human–agent collaboration.
Declarative operator
Pandas
PySpark
scan({ path } , { options })
pd.read_csv(...)
spark.read.csv(...)
filter({ condition })
query(...)
filter(...)
distinct({ columns })
drop_duplicates(...)
dropDuplicates(...)
sort({ columns } , { orders })
sort_values(...)
orderBy(...)
limit({ offset } , { count })
iloc[...]
offset(...).limit(...)
project({ mapping })
[[...]].rename(...)
select(...)
Table 2. Declarative operators and example implementations.
Figure 3. A dataflow for ML model training and inference, with a corresponding physical plan.
Figure 4. System architecture of BobFlow.
Figure 5. Dataflow ReAct life cycle for a data science task.
Action
Specifies
Application
DescribeReasoning
Reasoning
Display reasoning in chat.
AnswerTask
Answer
Display answer in chat.
GetDataflow
Operators, edges
Inspect operators, schemas, and edges.
SetDataflow
Dataflow changes
Add, delete, or update operators and edges.
GetCode
v∈V
Retrieve registered physical implementation.
SetCode
Code
Register implementation for custom operators.
Table 3. Actions for solving tasks using dataflows.
Figure 6. Assemble, Think, and Act in Dataflow ReAct.
Figure 7. Human interactions in Dataflow ReAct.
Figure 8. Full output versus samples and result statistics.
Figure 9. Pruning irrelevant observations for u at v3 .
Figure 10. User study results for Scenarios 1 (S1) and 2 (S2).
Dimension
Questionnaire item
Script ReAct
BobFlow
Δ
Intuitiveness
Easy to collaborate with the agent
3.9
4.0
+0.1
Transparency
Easy to grasp processed/produced tables
3.5
4.3
+0.8
Confident about what each step does
3.3
3.8
+0.5
Easy to view each step’s output tables
2.6
4.2
+1.6
Easy to follow data lineage
2.3
4.2
+1.9
Clear final-answer vs. exploratory steps
2.9
4.0
+1.1
Table 4. Participant ratings of intuitiveness, transparency, and control (1 = strongly disagree; 5 = strongly agree). Means are shown; bold indicates the more favorable score; Δ = BobFlow − Script ReAct.
Benchmark
Agent
GPT-5.6 Luna
GPT-5.6 Terra
GPT-5.6 Sol
GPT-6.1 Sol
Acc.
Cost
Acc.
Cost
Acc.
Cost
Acc.
Cost
KramaBench
Script ReAct
62.3
0.40
64.3
6.99
68.9
10.22
78.9
3.22
BobFlow
76.4
0.30
72.3
2.76
76.4
5.09
78.4
1.67
PrepBench
Script ReAct
69.9
1.21
71.9
24.56
79.4
45.97
90.5
17.33
BobFlow
83.3
0.99
86.3
8.67
88.6
16.50
90.8
6.33
Benchmark
Agent
Gemini 3.8 Flash
Gemini 4 Argon
DeepSeek V4.1-Flash
Claude Opus 5.5
Table 5. Task accuracy scores and inference cost ($); bold indicates the better value (higher accuracy, lower cost).
Figure 11. Input and output tokens per benchmark run.
Model
Accuracy
Δ Acc.
Cost ($)
Δ Cost
GPT-5.6 Luna
82.4 → 83.3
+0.9
1.21 → 0.99
−18.2 %
GPT-5.6 Terra
85.9 → 86.3
+0.4
10.66 → 8.67
−18.7 %
GPT-5.6 Sol
88.2 → 88.6
+0.4
21.35 → 16.50
−22.7 %
GPT-6.1 Sol
88.6 → 90.8
+2.2
8.39 → 6.33
−24.6 %
Gemini 3.8 Flash
90.2 → 89.5
−0.7
16.19 → 13.22
−18.3 %
Gemini 4 Argon
90.2 → 91.5
+1.3
17.97 → 15.42
−14.2 %
Table 6. Data-property optimization on PrepBench: BobFlow without opt1 → BobFlow. Accuracy changes are absolute; cost changes are relative.
Figure 12. Simplified traces of four KramaBench cases comparing BobFlow and Script ReAct.
Figure 13. Median context size (top) and cache-miss rate (bottom) versus the number of preceding interactions.
Model
Configuration
Accuracy
Input
Output
Cost ($)
(K)
(K)
Cold
Warm
GPT-5.6 Luna
BobFlow without opt2
42.9
126
1.4
9.73
3.54
BobFlow
42.9
54
1.9
5.43
4.85
GPT-5.6 Terra
BobFlow without opt2
54.6
99
1.2
88.33
29.52
BobFlow
58.4
45
1.5
49.37
43.47
DeepSeek V4.1-Flash
BobFlow without opt2
58.0
239
4.0
9.87
2.57
Table 7. LongDS-Bench v1.1 Lite results; mean tokens in thousands per interaction.
Figure 14. Memory growth from interaction 100 onward.
Figure 15. Jump latency (left) and memory with and without tracking (right) in replays of over 5,000 snapshots.