Turnslide: Scalable Multi-Turn Data Synthesis by Walking a Finite-State Machine

Aaron Fainman; Gabriela Kadlecová; Maciej Gryka; Bartosz Kruszczyński; Usman Zafar; Cédric Archambeau; Aaron Klein; David Salinas

cs.CLOct 5, 2026

Turnslide: Scalable Multi-Turn Data Synthesis by Walking a Finite-State Machine

Authors: Aaron Fainman, Gabriela Kadlecová, Maciej Gryka, Bartosz Kruszczyński, Usman Zafar, Cédric Archambeau, Aaron Klein, David Salinas, +2 more

Organizations: Imperial College London · distil labs · Agon · ELLIS Institute Tübingen

Abstract

Small language models are inexpensive to serve and can run on private infrastructure, but base models are often not good enough at multi-turn tool calling, and fine-tuning them needs per-API data that rarely exists. Existing synthesis methods are too expensive for high-scale fine-tuning, as they often require mock operational environments for different domains and multiple LLM calls per generated conversation turn. We introduce a fully automated, lightweight synthesis framework that models each API as a finite-state machine, representing the system as abstract states that determine when each tool may be called, producing state-valid sequences of tools; sequences are translated into complete examples with a single LLM call. Rather than optimize diversity, we set a target distribution over the number of turns, the tool sequence and task complexity. We measure data quality by fine-tuning SLMs on generated trajectories, showing that our FSM-based generation significantly improves downstream accuracy over an unmutated baseline and, against existing works, reaches 70.7% full accuracy over 63.4% and 53.7% with 3.6-6.6 $\times$ fewer tokens.

Figures & tables

Figure 1: Overview of our end-to-end pipeline showing offline initialization of the FSM, assembly of the blueprint by sampling specific attributes, generation of conversation trajectories in a single pass, and verification of examples. Downstream performance is measured after fine-tuning an SLM.

Method	Summary		Trading		Travel		VehicleControl		FileSystem 1		Tick.&Trav. 2
Method	$\beta(\pm)SE$	$p$	Per Turn	Full	Per Turn	Full	Per Turn	Full	Per Turn	Full	Per Turn	Full
Base Student	—	—	42.7	26.3	48.0	21.5	38.5	5.6	49.4	18.9	34.3	12.5
Baseline	—	—	76.9	43.9	79.6	47.8	69.4	18.8	80.7	67.3	78.1	38.5
Turns Mutator	2.89 $\pm$ 2.87	0.321	76.0	44.9	76.5	42.3	64.3	19.1	88.9	74.2	76.2	40.4
Markovian	7.88 $\pm$ 3.19	0.03	76.8	41.5	83.1	60.6	70.3	19.7	94.9	84.5	75.9	36.6
FSM	10.1 $\pm$ 2.62	$<$ 0.01	81.5	54.2	84.2	61.5	72.6	22.5	94.5	80.2	78.2	47.2

Table 1: Fine-tuning performance across different generation conditions showing the per-turn accuracy (%), the full conversation accuracy (%), regression coefficient

\beta

and the standard error (%) of the generation condition and the statistical significance

p

of the result.

Method	Calls/ Trajectory	Tokens/ Trajectory	Tokens/ Turn	Examples/ 20M Tokens	Fine-Tune Full Acc.(%)	Fine-Tune Per Turn Acc. (%)	LLM Eval (/5)
ToolFlow	34.3	99,846	22,327	200	63.4	85.7	3.03
APIGen-MT	107.2	184,304	35,943	109	53.7	80.1	3.04
Ours	1.0	27,724	5,545	721	70.7	87.2	3.34

Table 2: Cost and Performance Results compared with other Methods

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 2: Distribution of tools in the generated data for the TradingBot API with and without the tool sampling procedure described.

Method	Model	Trading		Travel		VehicleControl		FileSystem 1		Tick.&Trav. 2
Method	Model	Per Turn	Full	Per Turn	Full	Per Turn	Full	Per Turn	Full	Per Turn	Full
Base Student	Qwen	79.9	52.6	62.3	30.0	53.9	9.9	68.7	25.3	55.6	23.1
Base Student	Llama	5.5	0.0	33.6	13.0	23.1	1.3	30.0	12.5	13.0	1.9
Baseline	Qwen	76.0	39.0	74.8	38.5	64.9	10.7	82.3	65.5	72.6	30.7
Baseline	Llama	77.7	48.7	84.4	57.1	73.8	26.9	79.1	69.0	83.5	46.2
Turns Mutator	Qwen	73.8	38.3	76.5	42.3	64.1	13.1	89.2	72.4	69.6	30.8

Appendix

Table 3: Fine-tuning performance across generation conditions for both student models, showing per-turn accuracy (%) and full-conversation accuracy (%).

Explore similar work

Data Turnstile: A Scalable Open Framework for Function-Calling Data Generation
Jul 31, 2026Goutham Ramakrishnan, Megha SharmaLarge Language Model Tool Use Agentic Benchmarks
Amazon AGI Bengaluru, India
ToolLoop: Closed-Loop Tool-Use Data Synthesis via Decomposed Generation and Dynamic Self-Feedback
Sep 8, 2026Min Zeng, Yuzhou Liu, Zhenyu Cao +5Loop Decomposition
vivo AI Lab
State-Grounded Multi-Agent Synthetic Data Generation for Tool-Augmented LLMs
Jun 15, 2026Rahul Khedar, Eshita, Sneha Teja Sree Reddy Thondapu +10Synthetic Data
PayPal AI

Jul 31, 2026, cs.CLJ/K move · Enter open · S save

Data Turnstile: A Scalable Open Framework for Function-Calling Data Generation

Goutham Ramakrishnan, Megha Sharma

Amazon AGI Bengaluru, India

Large Language Model Tool Use Agentic Benchmarks Synthetic Training Data Data Generation Application Programming Interfaces

Open paper AskPDF arXiv

Small language models (SLMs) are attractive for agentic deployment due to low latency, reduced cost, and on-device privacy, yet they struggle with tool-use tasks where training data is scarce and noisy. Unlike larger models, SLMs cannot compensate for low-quality supervision through sheer capacity, making data quality the critical bottleneck. We present Data Turnstile, an open-source framework that takes user-defined API specifications and generates high-quality synthetic training data for function calling. Turnstile decomposes multi-turn tool-use interactions into constrained, stepwise generation with validation and error-feedback loops, providing fine-grained control over API diversity, conversation complexity, and output correctness. We demonstrate effectiveness of domain adaptation with Turnstile data on two challenging function calling benchmarks. On the BFCL single-turn benchmark, a Qwen3-0.6B fine-tuned on Turnstile data without chain-of-thought achieves 75.9% overall accuracy (versus 67.4% for the base model with thinking enabled), closing the gap with thinking-enabled Qwen3-1.7B (78.4%) and Qwen3-4B (79.9%) despite being 3

\times

and 7

\times

smaller respectively. On

τ^2

-bench, a multi-turn agentic benchmark, Turnstile-trained Qwen3-1.7B achieves 31.1% pass^1 on the Telecom domain, improving 4.7

\times

over its 6.6% base and surpassing Qwen2.5-32B-Instruct (27.4%), a model 19

\times

larger. Turnstile-trained Qwen3-0.6B achieves 24.6%, improving 7

\times

over its 3.5% base and approaching the 32B model (53

\times

larger). We release Data Turnstile along with a dataset spanning 1,000+ APIs and 100K+ multi-turn interactions.

Abstract

Explore similar work

Data Turnstile: A Scalable Open Framework for Function-Calling Data Generation

ToolLoop: Closed-Loop Tool-Use Data Synthesis via Decomposed Generation and Dynamic Self-Feedback

State-Grounded Multi-Agent Synthetic Data Generation for Tool-Augmented LLMs