TOOLCAD: Exploring Tool-Using Large Language Models in Text-to-CAD Generation with Reinforcement Learning

Yifei Gong; Xing Wu; Wenda Liu; Kang Tu

cs.CVApr 9, 2026

TOOLCAD: Exploring Tool-Using Large Language Models in Text-to-CAD Generation with Reinforcement Learning

Authors: Yifei Gong, Xing Wu, Wenda Liu, Kang Tu

Organizations: School of Computer Engineering & Science, Shanghai University

Abstract

Computer-Aided Design (CAD) is an expert-level task that relies on long-horizon reasoning and coherent modeling actions. Large Language Models (LLMs) have shown remarkable advancements in enabling language agents to tackle real-world tasks. Notably, there has been no investigation into how tool-using LLMs optimally interact with CAD engines, hindering the emergence of LLM-based agentic text-to-CAD modeling systems. We propose ToolCAD, a novel agentic CAD framework deploying LLMs as tool-using agents for text-to-CAD generation. Furthermore, we introduce an interactive CAD modeling gym to rollout reasoning and tool-augmented interaction trajectories with the CAD engine, incorporating hybrid feedback and human supervision. Meanwhile, an end-to-end post-training strategy is presented to enable the LLM agent to elicit refined CAD Modeling Chain of Thought (CAD-CoT) and evolve into proficient CAD tool-using agents via online curriculum reinforcement learning. Our findings demonstrate ToolCAD fills the gap in adopting and training open-source LLMs for CAD tool-using agents, enabling them to perform comparably to proprietary models, paving the way for more accessible and robust autonomous text-to-CAD modeling systems.

Figures & tables

Figure 1: Prompt-to-Tool vs. Prompt-to-Code . L3 expert-level modeling text enables CAD tool-using agents to plan, reason, call tools, and complete modeling tasks via iterative CAD-engine feedback.

Figure 2: CAD-specific Tool-Using Agent Workflow for Text-to-CAD Generation . Given an expert-level natural language-based CAD design intent, the ToolCAD framework performs 1) modeling decision-making, 2) equip with modeling tools and environment, 3) automatic modeling with reflection.

Figure 3: Online-RL framework of ToolCAD . Starting from human-supervised CAD trajectories, the policy is refined into a robust agent through online curriculum reinforcement learning. The ORM training is consistent with human-supervised knowledge.

Figure 4: Evaluation of ToolCAD with RL. Modeling success rate, tool-calling accuracy, and geometric precision ( $1-\textit{IoU}$ ) are evaluated to comprehensively measure the tool-using agent’s modeling behavior and performance.

Frontier LLMs
Model	Method	IR $\downarrow$	MCD $\downarrow$	Avg. $S_{R}$ (%) $\uparrow$
GPT-4o	Zero-shot-CoT	20.49	19.84	48.4
GPT-4o	ReAct	9.12	5.88	62.7
Qwen3-235B-A22B	Zero-shot-CoT	30.25	37.18	51.3
Qwen3-235B-A22B	ReAct	25.65	29.83	60.5
Open-Source LLMs

Table 1: Main comparisons of average CAD modeling success rate(SR) and quality across different baselines. The best and second-best models are highlighted.

Method	Fine-Tuning	VLM-Based	Backbone	Prompt Type	CAD-Code	ACC ${}_{T}\uparrow$	COV $\uparrow$	MMD $\downarrow$	JSD $\downarrow$
CAD-Assisant Mallis et al. (2024)	$\times$	$\checkmark$ (GPT-4o)	GPT-4o	L0 text + docstr + image	$\checkmark$	51.25	60.32	6.13	25.09
CAD-GPT Wang et al. (2025c)	$\checkmark$	$\checkmark$ (LLaVA)	LLaVA	L0 text + image	$\times$	52.78	64.59	4.49	22.17
CAD-Llama Li et al. (2025)	$\checkmark$	$\checkmark$ (CLIP)	LLaMA3-8B-Instruct	L0 $\sim$ L1 text	$\checkmark$	79.46	74.86	1.62	3.85
VideoCAD Man et al. (2025)	$\checkmark$	$\checkmark$ (DINOv2/CLIP)	ViT	video frame	$\times$	—	49.25	11.75	32.19
FlexCAD Zhang et al. (2025)	$\checkmark$	$\times$	LLaMA3-8B-Instruct	L3 text token	$\times$	81.43	70.49	2.35	4.79
CADCodeVerify Alrashedy et al. (2025)	$\times$	$\checkmark$ (GPT-4/Gemini-1.5-Pro)	GPT-4/Gemini-1.5-Pro/CodeLlama	L2 $\sim$ L3 text	$\checkmark$	—	64.37	5.46	18.26

Table 2: Comparison of agentic CAD generation methods using vision-language models (VLMs) and large language models (LLMs) across multi-part task with varying complexity.

Figure 5: Case study of tool-using agent modeling trajectories. Our ORM (100 $\%-$ trained) detects subtle step-level near-miss failures such as " Unit Missing " and " Boolean Error " that are visually indistinguishable to the VLM judger (Qwen2.5-VL).

Methods	Feedback Source	1P	3P	5+P
CAD-Assistant (GPT-4o)	Visual+Docstr	82.5	59.1	20.8
CAD-GPT (LLaVA)	Visual+Text	80.3	54.8	18.4
TOOLCAD-VLM (variant)	Visual+Text	85.6	50.3	15.9
TOOLCAD (Ours)	Text	87.2	69.3	29.7

Table 3: VLM-based feedback vs. Engine-based feedback success rate (%)

Methods	Instruction-level	COV $\uparrow$	JSD $\downarrow$	Novelty $\uparrow$	IoU best $\uparrow$
Text2CAD	@L0	40.27%	36.49	94.33%	40.51%
	@L1	49.35%	20.21	92.58%	45.39%
	@L2	54.68%	17.15	90.53%	54.77%
	@L3	61.94%	10.28	88.47%	61.24%
TOOLCAD (Ours)	@L0	58.19%	20.81	98.14%	46.63%
	@L1	61.17%	11.25	96.31%	53.91%

Table 4: Comparison on geometry quality and diversity across different instruction levels.

Figure 6: Ablation study of online RL framework on step-level feedback and part-wise curriculum learning strategy.

Method	Instruction-level
Method	@L1		@L2		@L3
	Sketch	Extrusion	Sketch	Extrusion	Sketch	Extrusion
Text2CAD	30.35	57.19	47.25	69.31	58.47	88.62
ToolCAD (Qwen2.5-7B)	24.59	44.22	43.23	75.26	78.91	93.44
ToolCAD (Qwen3-8B)	27.58	49.25	45.69	77.86	81.52	95.21

Table 5: Instruction-level Sketch and Extrusion F1 score.

	Our ORM	GPT-4o	Qwen3-235B	Qwen2.5-VL
Test Cases (%)	82.7	75.9	70.4	55.3
Hard Cases (%)	65.2	54.7	50.8	41.6

Table 6: Performance of ORMs.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Number of Part	#Train Traj.	#Test Traj.
Part-1	100	40
Part-2	130	40
Part-3	300	40
Part-4	130	40
Part- $\geq$ 5	122	40
Total	782	200

Appendix

Table 7: The statistics of held-in modeling tool-using trajectories.

Figure 7: Comparison of different training strategies.

Figure 8: Average perplexity vs. threshold factor $\alpha$ during online exploration.

Model	Avg Tokens	Avg Latency (ms)
ToolCAD (Qwen2.5-7B)	5178	648
ToolCAD (Qwen3-7B)	6493	788
GPT-4o	11703	982
Qwen3-235B-A22B	9820	1108
CAD-Assisant(GPT-4o)	30802	1326

Appendix

Table 8: Comparison of average token usage per tool-call and average tool-call latency across different models.

Figure 9: Scaling Laws of CAD Tool-using Agents.

Figure 10: ToolCAD’s Agent Modeling Examples

Figure 11: Integration of MCP provides structured environment and tool management, which enhances the extensibility and flexibility of CAD modeling tools.

\delta_{\ell}^{\prime}=\frac{1}{|\mathcal{V}_{\ell}|}\sum_{\tau\in\mathcal{V}_{\ell}}\exp\left(-\frac{1}{|\tau|}\sum_{k=1}^{|\tau|}\log\pi_{\theta}(a_{k}|s_{k})\right)

Appendix

Algorithm 1 Online Curriculum RL for CAD Tool-Using Agents

Text-to-CAD Generation

Text-to-CAD generation turns natural-language design intent into executable, editable parametric 3D models and assemblies.

See all 48 papers

PR-CAD: Progressive Refinement for Unified Controllable and Faithful Text-to-CAD Generation with Large Language Models
Mar 27, 2026Jiyuan An, Jiachen Zhao, Fan Chen +6Text-to-CAD Generation Generative Modeling
National Language Resources Monitoring and Research Center for Print Media, Beijing Language and Culture University, China · School of Information Science, Beijing Language and Culture University, China · School of Computer Science and Technology, Beijing Jiaotong University, China +3
RA-CAD: Learning Post-Execution Critique for State-Aware Text-to-CAD Generation
Aug 6, 2026Shuhao Yan, Changhao He, Peng Hu +1RL for Code Generation Code Generation
College of Computer Science, Sichuan University, Chengdu, China · School of Artificial Intelligence, Sichuan University, Chengdu, China · National Key Laboratory of Fundamental Algorithms and Models for Engineering Numerical Simulation, Sichuan University, Chengdu, China
Memory-Augmented Reinforcement Learning Agent for CAD Generation
May 19, 2026Yin Xiaolong, Liu Yu, Shen Jiahang +4Text-to-CAD Generation RL for Generative Models
Zhejiang University · Ocean University of China

Mar 27, 2026J/K move · Enter open · S save

PR-CAD: Progressive Refinement for Unified Controllable and Faithful Text-to-CAD Generation with Large Language Models

Jiyuan An, Jiachen Zhao, Fan Chen, Liner Yang+5

National Language Resources Monitoring and Research Center for Print Media, Beijing Language and Culture University, China · School of Information Science, Beijing Language and Culture University, China · School of Computer Science and Technology, Beijing Jiaotong University, China+3

Text-to-CAD Generation Generative Modeling Parametric CAD Modeling

Open paper AskPDF arXiv

The construction of CAD models has traditionally relied on labor-intensive manual operations and specialized expertise. Recent advances in large language models (LLMs) have inspired research into text-to-CAD generation. However, existing approaches typically treat generation and editing as disjoint tasks, limiting their practicality. We propose PR-CAD, a progressive refinement framework that unifies generation and editing for controllable and faithful text-to-CAD modeling. To support this, we curate a high-fidelity interaction dataset spanning the full CAD lifecycle, encompassing multiple CAD representations as well as both qualitative and quantitative descriptions. The dataset systematically defines the types of edit operations and generates highly human-like interaction data. Building on a CAD representation tailored for LLMs, we propose a reinforcement learning-enhanced reasoning framework that integrates intent understanding, parameter estimation, and precise edit localization into a single agent. This enables an "all-in-one" solution for both design creation and refinement. Extensive experiments demonstrate strong mutual reinforcement between generation and editing tasks, and across qualitative and quantitative modalities. On public benchmarks, PR-CAD achieves state-of-the-art controllability and faithfulness in both generation and refinement scenarios, while also proving user-friendly and significantly improving CAD modeling efficiency.

Abstract

Text-to-CAD Generation

Explore similar work

PR-CAD: Progressive Refinement for Unified Controllable and Faithful Text-to-CAD Generation with Large Language Models

RA-CAD: Learning Post-Execution Critique for State-Aware Text-to-CAD Generation

Memory-Augmented Reinforcement Learning Agent for CAD Generation