GenesisFunc: Multi-Agent Data Generation for Accurate and Generalizable Function-Calling

Hao-Xiang Xu; Chong Deng; Jiaqing Liu; Wen Wang; Qian Chen; Lujia Bao; Xiangang Li; Zhen-Hua Ling

cs.CLApr 10, 2026

GenesisFunc: Multi-Agent Data Generation for Accurate and Generalizable Function-Calling

Authors: Hao-Xiang Xu, Chong Deng, Jiaqing Liu, Wen Wang, Qian Chen, Lujia Bao, Xiangang Li, Zhen-Hua Ling

Organizations: Tongyi Fun Team, Alibaba Group

Abstract

Large Language Models (LLMs) extend their capabilities through function-calling (FC), which relies on training data with high quality, diversity, and broad coverage of scenario. However, obtaining and annotating real function-calling data is challenging, while synthetic data from existing pipelines often suffers from unreliable APIs, limited tool scalability, insufficient diversity, and weak quality control. To address these, we present GenesisFunc, an automated pipeline for generating FC training data. Starting from reliable tools in widely used public benchmarks, our GenesisFunc employs a multi-agent framework to support a dialogue generation system that produces conversations spanning diverse scenarios, while maintaining both diversity and quality throughout the process. The accuracy of the data is further reinforced through a multi-stage evaluation system. We fine-tune an 8B LLM on the synthetic dataset and show through extensive experiments that it outperforms similarly sized open-source models in in-domain FC performance and out-of-domain generalization, while reaching FC capabilities comparable to some of the latest API-based models. In addition, our method demonstrates strong potential to scale effectively across downstream tools, underscoring its real-world applicability.

Figures & tables

Figure 1: The overall framework of GenesisFunc , consisting of a Tool Pool built from reliable tools in open-source benchmarks, a Multi-Agent Dialogue Generation system, and a Multi-Stage Evaluation process.

	Non-Live					Live
Models	Simple	Multiple	Parallel	Parallel	Overall	Simple	Multiple	Parallel	Parallel	Overall
				Multiple					Multiple
GPT-4o	76.50	91.00	90.00	78.00	83.88	70.54	70.75	62.50	66.67	70.54
GPT-4o-mini	80.33	92.00	89.50	90.50	88.08	79.46	76.26	87.50	70.83	76.91
Gemini-2.5-Pro	78.67	94.00	93.50	92.00	89.54	85.66	74.36	87.50	83.33	76.83
LLaMA-3.1-8B-Instruct	71.00	95.00	87.50	82.50	84.00	72.87	71.13	50.00	45.83	59.96

Table 1: Accuracy on the In-Domain BFCL dataset. GenesisFunc -8B is fine-tuned on Qwen3-8B using training data generated by our pipeline. The best results in each category are in bold and the second best results are underlined . We report mean and standard deviation (

\pm

SD) of the Overall accuracy based on three independent runs.

	API-Bank				ACEBench
					Normal						Special
Models	Level-1	Level-2	Level-3	Overall	Atom	Single-Turn	Multi-Turn	Similar API	Preference	Overall	Overall
GPT-4o	76.19	42.96	35.21	51.45	90.00	78.00	77.00	85.00	83.00	82.60	87.60
GPT-4o-mini	74.69	45.93	40.77	53.80	84.33	73.50	66.50	77.00	78.00	75.87	79.90
Gemini-1.5-Pro	75.43	43.26	41.51	53.40	84.50	76.80	64.50	80.00	78.00	76.76	79.00
Qwen3-8B	71.68	52.24	42.75	55.56	80.00	65.50	51.00	68.00	60.00	64.90	76.67

Table 2: Accuracy on the Out-of-Domain API-Bank and ACEBench datasets. The best results in each category are in bold and the second best results are underlined . We report mean and standard deviation (

\pm

SD) of the Overall accuracy from our model based on three independent runs.

Panel of Figure 2 — Figure 2: Ablation study of the proposed agents. We separately remove the Judge Agent and the Memory Agent, and evaluate our GenesisFunc -8B on BFCL in (a) Non-Live and (b) Live settings.

Panel of Figure 3 — Figure 3: Ablation study on the number of samples. We control the number of dialogues generated per run and evaluate on BFCL in (a) Non-Live and (b) Live settings.

	Normal						Special
Models	Atom	Single-Turn	Multi-Turn	Similar API	Preference	Overall	Overall
Qwen3-8B	80.00	65.50	51.00	68.00	60.00	64.90	76.67
GenesisFunc -8B	88.00	77.00	65.00	74.00	64.00	73.60	83.67
GenesisFunc -8B-RL(all)	87.00	78.00	66.00	72.00	62.00	73.00	82.45
GenesisFunc -8B-RL(part)	88.00	79.00	70.00	75.00	64.00	75.20	82.88

Table 3: Accuracy on ACEBench using reinforcement learning . The best overall results in each category are marked in bold. The second best results are underlined .

Models	BFCL	API-Bank	ACEBench
Qwen3-8B	83.20	55.56	70.79
ToolACE-8B	85.89	56.21	35.65
GenesisFunc	88.55	64.79	78.64
+ ACEBench	87.89	65.11	81.87

Table 4: Overall Accuracy on BFCL, API-Bank and ACEBench adding the tools in ACEBench. The best overall results in each category are marked in bold. The second best results are underlined .

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: The system prompts of each agent in multi-agent framework.

Aspect	Rules
Tool Definition Completeness	Check that the tool schema parses and includes required fields (name, description, parameters).
	Verify that parameter specs declare type or format and required or optional flags.
	Ensure the tool name is unique within the tools pool.
Call Format & Parameter Compliance	Confirm the called tools exist in the tools pool and use the correct name.
	Ensure all required parameters appear exactly once and no unknown keys exist.
	Validate that values satisfy declared type/regex/range constraints (e.g., ISO date, enum).

Appendix

Table 5: Example rules for our Rule Checker in multi-stage evaluation.

Dataset	Single-Single	Single-Multi	Multi-Single	Multi-Multi	Special-Case
Gorilla Patil et al. (2024)	✓	$\times$	$\times$	$\times$	$\times$
ToolAlpaca Tang et al. (2023)	✓	✓	$\times$	$\times$	$\times$
ToolLLM Qin et al. (2024)	✓	✓	$\times$	$\times$	$\times$
xLAM Liu et al. (2024)	✓	✓	✓	$\times$	$\times$
ToolACE Liu et al. (2025a)	✓	✓	✓	$\times$	✓
GenesisFunc	✓	✓	✓	✓	✓

Appendix

Table 6: Comparison of dialogue scenario coverage across representative tool-augmented datasets. Here, Single-Single denotes single-turn dialogues designed to accomplish a single task. Single-Multi refers to single-turn dialogues that involve multiple tasks within the same interaction. Multi-Single indicates multi-turn dialogues focused on a single task. Multi-Multi represents multi-turn dialogues that cover multiple tasks. and Special-Case corresponds to challenging scenarios such as the absence of applicable tools or cases where parameter filling is infeasible. Our dataset uniquely covers all five categories.

Figure 5: Distribution of slot filling ratios across 400 sampled tools.

Learning Rate	Warmup Ratio	LR Scheduler	Batch Size	Epochs	LoRA Rank	LoRA Alpha
$10^{-4}$	0.1	cosine	48	3	16	32

Appendix

Table 7: Configuration of hyper-parameters for model training.

	Non-Live					Live
Models	Simple	Multiple	Parallel	Parallel	Overall	Simple	Multiple	Parallel	Parallel	Overall
				Multiple					Multiple
Qwen3-8B	78.92	95.00	91.50	89.00	88.60	80.23	77.21	81.25	75.00	77.79
ToolACE-8B	81.17	96.00	94.00	93.00	91.04	82.95	79.58	75.25	85.12	80.73
GenesisFunc -8B(dynamic)	86.85	97.25	96.15	93.00	93.31	88.25	80.50	81.50	84.88	83.78
GenesisFunc -8B(max)	86.00	96.85	95.50	92.25	92.65	86.80	80.00	81.25	83.86	82.98

Appendix

Table 8: Ablation study of different slot selection strategies in the function agent. To compare with our proposed dynamic slot selection approach, we also adopt two fixed strategies: filling all optional parameters and leaving them empty, and evaluate on BFCL. The best results are marked in bold, and the second best are underlined .

Figure 6: General abilities of models. Evaluation are conducted in five dimensions.

	Non-Live					Live
Models	Simple	Multiple	Parallel	Parallel	Overall	Simple	Multiple	Parallel	Parallel	Overall
				Multiple					Multiple
Qwen3-8B	78.92	95.00	91.50	89.00	88.60	80.23	77.21	81.25	75.00	77.79
ToolACE-8B	81.17	96.00	94.00	93.00	91.04	82.95	79.58	75.25	85.12	80.73
GenesisFunc -8B	86.85	97.25	96.15	93.00	93.31	88.25	80.50	81.50	84.88	83.78
- multi-stage evaluation module	85.17	97.00	96.00	92.50	92.68	87.00	80.00	81.25	84.05	83.08

Appendix

Table 9: Ablation study of multi-stage evaluation module. We remove the multi-stage evaluation module, and evaluate on BFCL. The best results in each category are marked in bold. The second best results are underlined .

Panel of Figure 7 — Figure 7: Scaling analysis of function-calling performance. We evaluate raw and fine-tuned Qwen-3-xB models of different sizes on the BFCL benchmark in (a) Non-Live and (b) Live settings.

Panel of Figure 8 — Figure 8: Generalization across model backbones. We evaluate raw and fine-tuned LLMs with different backbone architectures on the BFCL benchmark in (a) Non-Live and (b) Live settings.

Figure 9: Example of a single-task scenario within single-turn dialogue.

Figure 10: Example of a multi-task scenario within single-turn dialogue.

Figure 11: Example of a single-task scenario within multi-turn dialogue.

Figure 12: Example of a multi-task scenario within multi-turn dialogue.

Figure 13: Example of a special-case dialogue.

Synthetic Data Generation for LLM Agents

See all 28 papers

Data Turnstile: A Scalable Open Framework for Function-Calling Data Generation
Jul 31, 2026Goutham Ramakrishnan, Megha SharmaTool-Augmented Language Model Agents Small Language Models
Amazon AGI Bengaluru, India
ToolLoop: Closed-Loop Tool-Use Data Synthesis via Decomposed Generation and Dynamic Self-Feedback
Sep 8, 2026Min Zeng, Yuzhou Liu, Zhenyu Cao +5Tool-Augmented Language Model Agents Synthetic Data Generation
vivo AI Lab
Firefly: Illuminating Large-Scale Verified Tool-Call Data Generation from Real APIs
May 17, 2026Yuxuan Lu, Ziyi Wang, Yingzhou Lu +12Tool-Augmented Language Model Agents Synthetic Data Generation
Northeastern University · Independent Researcher

Jul 31, 2026J/K move · Enter open · S save

Data Turnstile: A Scalable Open Framework for Function-Calling Data Generation

Goutham Ramakrishnan, Megha Sharma

Amazon AGI Bengaluru, India

Tool-Augmented Language Model Agents Small Language Models LLM Tool Use Synthetic Data Generation for Language Models Tool Use Synthetic Data Generation for LLM Agents

Open paper AskPDF arXiv

Small language models (SLMs) are attractive for agentic deployment due to low latency, reduced cost, and on-device privacy, yet they struggle with tool-use tasks where training data is scarce and noisy. Unlike larger models, SLMs cannot compensate for low-quality supervision through sheer capacity, making data quality the critical bottleneck. We present Data Turnstile, an open-source framework that takes user-defined API specifications and generates high-quality synthetic training data for function calling. Turnstile decomposes multi-turn tool-use interactions into constrained, stepwise generation with validation and error-feedback loops, providing fine-grained control over API diversity, conversation complexity, and output correctness. We demonstrate effectiveness of domain adaptation with Turnstile data on two challenging function calling benchmarks. On the BFCL single-turn benchmark, a Qwen3-0.6B fine-tuned on Turnstile data without chain-of-thought achieves 75.9% overall accuracy (versus 67.4% for the base model with thinking enabled), closing the gap with thinking-enabled Qwen3-1.7B (78.4%) and Qwen3-4B (79.9%) despite being 3

\times

and 7

\times

smaller respectively. On

τ^2

-bench, a multi-turn agentic benchmark, Turnstile-trained Qwen3-1.7B achieves 31.1% pass^1 on the Telecom domain, improving 4.7

\times

over its 6.6% base and surpassing Qwen2.5-32B-Instruct (27.4%), a model 19

\times

larger. Turnstile-trained Qwen3-0.6B achieves 24.6%, improving 7

\times

over its 3.5% base and approaching the 32B model (53

\times

larger). We release Data Turnstile along with a dataset spanning 1,000+ APIs and 100K+ multi-turn interactions.

Abstract

Synthetic Data Generation for LLM Agents

Explore similar work

Data Turnstile: A Scalable Open Framework for Function-Calling Data Generation

ToolLoop: Closed-Loop Tool-Use Data Synthesis via Decomposed Generation and Dynamic Self-Feedback

Firefly: Illuminating Large-Scale Verified Tool-Call Data Generation from Real APIs