DeciCAD: Harness-Grounded Learning of Action Policies for CAD Agents

Yufeng Peng; Yixin Zhang; Ziyi Liu; Zhikun Xu; Jiarong Yang; Sijie Wang; Wenjie Feng; Qiyu Kang

cs.AIOct 15, 2026

DeciCAD: Harness-Grounded Learning of Action Policies for CAD Agents

Authors: Yufeng Peng, Yixin Zhang, Ziyi Liu, Zhikun Xu, Jiarong Yang, Sijie Wang, Wenjie Feng, Qiyu Kang

Organizations: University of Science and Technology of China · Nanyang Technological University

Abstract

Large Language Model agents operate through closed-loop action policies whose decision space is defined by an agent harness. We investigate how this operating space shapes agent behavior and policy learning in computer-aided design (CAD), a professional domain characterized by persistent state, geometric constraints, and long-horizon procedures. We instantiate this perspective in DeciCAD, a domain-native harness that exposes semantic CAD actions, decision-oriented observations, environment feedback, and persistent state transitions. Building on this harness, we develop a learning pipeline that extracts causal decision supervision from interaction trajectories and improves the policy through supervised fine-tuning (SFT) and geometry-grounded online reinforcement learning (RL). Across four CAD benchmarks, DeciCAD achieves the best F1 and intersection over union (IoU) while reducing model-output tokens by 45--88% relative to a general-purpose coding harness. Under matched supervision, its SFT policy improves F1 from 0.6883 to 0.7437 and outperforms the coding policy; subsequent online RL further raises geometry availability from 66.5% to 73.5% and F1 to 0.7880. These results demonstrate that the decision space induced by a harness materially affects both agent execution and policy learning in CAD.

Figures & tables

Figure 1: Harness-induced CAD decision spaces under controlled model and environment conditions. Given the same LLM policy, CAD task, persistent design state, and geometry evaluator, the coding harness represents decisions through code and shell commands and returns files and console output. DeciCAD instead exposes semantic CAD actions and returns object-level state and execution outcomes. The two interfaces therefore induce different action–observation histories over the same underlying modeling process.

Figure 2: Overview of DeciCAD . Semantic CAD actions are bound to real transitions in persistent CAD state, with bounded observations and execution feedback; the resulting trajectories support causal SFT and geometry-grounded online RL.

Figure 3: Semantic CAD Decision Bottleneck. Missing actions and state aliasing violate action and observation sufficiency, respectively; DeciCAD is designed to approach the decision-sufficient boundary. Method locations are conceptual and not to scale.

	CADPrompt				P3D-Text-to-3D				CADInstruct				P3D-Assembly-3D
Method	CD $\downarrow$	F1 $\uparrow$	IoU $\uparrow$	Avail. $\uparrow$	CD $\downarrow$	F1 $\uparrow$	IoU $\uparrow$	Avail. $\uparrow$	CD $\downarrow$	F1 $\uparrow$	IoU $\uparrow$	Avail. $\uparrow$	CD $\downarrow$	F1 $\uparrow$	IoU $\uparrow$	Avail. $\uparrow$
Zero-shot	0.32	61.84	74.10	93.50	0.35	37.11	48.15	90.00	0.93	19.28	23.24	100.00	0.52	25.40	22.35	71.92
CADSmith	0.26	63.70	72.08	100.00	0.33	40.74	52.73	100.00	0.85	20.11	23.14	100.00	0.64	26.27	21.19	96.06
CAD-Assistant	0.41	56.15	65.77	100.00	0.45	35.83	53.04	100.00	0.98	18.07	14.17	100.00	0.64	25.43	21.12	99.51
Claude Code	0.29	65.44	73.86	100.00	0.34	40.44	51.20	100.00	0.92	20.10	22.62	100.00	0.32	31.95	23.22	100.00
DeciCAD	0.28	65.83	75.12	100.00	0.25	43.63	54.22	100.00	0.84	23.49	23.35	100.00	0.49	32.11	27.07	100.00

Table 1: Overall performance across four benchmarks, ordered by increasing calibrated difficulty. CD, F1, and IoU are scaled by 100; Avail. is the percentage of tasks producing evaluable geometry. Best results are in bold and second-best results are underlined . Token counts in the lower block are averaged per task, are descriptive, and are not ranked.

Figure 4: Linear summaries of end-to-end F1 across difficulty quantiles. Dashed lines are least-squares fits over the seven quantiles.

Panel of Figure 5 — Figure 5: Interaction length versus cumulative output tokens. Each point is a complete task trajectory; dashed lines are log-linear fits.

Metric	SDO	SDO-CFF	RSO	EFO
Avail. (%) $\uparrow$	100.00	95.00	55.00	87.00
CD $\downarrow$	0.25	0.44	0.99	0.61
F1 $\uparrow$	43.63	32.26	21.62	31.75
IoU $\uparrow$	54.22	43.78	33.06	40.71
Input tokens $\downarrow$	390.54	3,050.00	3,450.00	3,200.00

Table 2: Ablation of observation and feedback design on P3D-Text-to-3D. Variants are Structured Decision-Oriented Observation (SDO; full), SDO with Coarse Failure Feedback (SDO-CFF), Raw State Observation (RSO), and Execution-Feedback-Only Observation (EFO). CD, F1, and IoU are scaled by 100; input tokens are in thousands per task.

Figure 6: Observable policy behavior across Base, SFT, and GRPO. The left and right panels show the Coding and DeciCAD harnesses. Columns report events per 100 decisions under a shared behavioral mapping; $n$ is the number of trajectories containing parseable action records.

Harness	Policy	Avail. (%)	F1	IoU	CD
Coding	Base	12.5	0.5902	0.5871	0.00972
Coding	SFT	60.5	0.6460	0.6286	0.01130
DeciCAD	Base	61.5	0.6883	0.6487	0.01023
DeciCAD	SFT	66.5	0.7437	0.6631	0.00782

Table 3: Closed-loop policy acquisition on the Text2CAD/DeepCAD test set. Availability is computed over all tasks; F1, IoU, and CD are computed over valid assets.

Harness	Policy	Avail. (%)	F1	IoU	CD
Coding	SFT	60.5	0.6460	0.6286	0.01130
Coding	SFT+GRPO	65.5	0.6253	0.5987	0.02253
DeciCAD	SFT	66.5	0.7437	0.6631	0.00782
DeciCAD	SFT+GRPO	73.5	0.7880	0.7036	0.00570

Table 4: Online policy improvement under matched training and evaluation. F1, IoU, and CD are computed over valid assets.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: The Semantic Decision Bottleneck compares harnesses by their decision capabilities while preserving optimal CAD performance.

Benchmark	Samples	Input	Task type
CADPrompt	200	Text	Part modeling
P3D-Text-to-3D	100	Text	Part modeling
CADInstruct	100	Text	Part modeling
P3D-Assembly-3D	203	Text + image	Assembly modeling

Appendix

Table 5: Statistics of the benchmarks used in our experiments.

Panel of Figure 8 — Figure 8: Mean end-to-end F1 across calibrated within-dataset difficulty quantiles.

Component	Setting
Student / teacher	Qwen3.5-9B / Claude Sonnet 4.6
SFT LoRA	rank 32, alpha 64, dropout 0.05
SFT learning rate	$1\times 10^{-5}$ , cosine
SFT context	65,536 tokens
Checkpoint interval	25 optimizer steps
GRPO LoRA	rank 32, alpha 64

Appendix

Table 6: Shared policy-learning configuration.

Panel of Figure 9 — Figure 9: Geometry–context trade-off across benchmarks. Each point reports mean F1 and cumulative model-visible input tokens for one method.

Figure 10: Calibrated difficulty distributions of the four benchmarks.

Figure 11: Allocation of observable decisions to Planning, Modeling, and Verification across datasets and difficulty quantiles. The upper and lower rows show Claude Code and DeciCAD , respectively.

Panel of Figure 12 — Figure 12: Qualitative comparison on P3D-Text-to-3D (left) and CADPrompt (right).

Harness	Audited tasks	Train / dev.	Target cap	Recovery targets
Coding	499	471 / 28	12	Up to 4 observed
DeciCAD	642	542 / 100	11 + 3	Up to 3 observed or controlled

Appendix

Table 7: SFT supervision obtained through the two harnesses. Both tracks use the same student model and optimization protocol.

Figure 13: Cross-harness policy trajectories and terminal geometry on the same CAD task. Rows show Coding SFT, Coding GRPO, DeciCAD SFT, and DeciCAD GRPO. Nodes summarize external calls and their execution status; the right side shows generated assets and the reference geometry. Unavailable indicates that no evaluable asset was produced.

Panel of Figure 14 — Figure 14: The project-level DeciCAD interface. The workspace keeps natural-language interaction, project resources, CAD skills, backend state, and streamed execution events in one persistent session. Users can inspect and control the runtime while semantic CAD actions continue to operate on the active document.

Figure 15: An editable CAD document produced through the FreeCAD backend. The viewport shows the generated solid, while the document tree retains the body, additive features, and underlying sketch created during the interaction. This persistent structure allows later inspection and modification through either DeciCAD or the native FreeCAD interface.

Domain	Canonical operations	Supported behavior
Document	DocumentCreate , DocumentOpen , DocumentSave , DocumentExport , DocumentClose , DocumentDeleteEntity	Create and open native FreeCAD documents; save editable documents; export selected targets to STEP or STL; close documents; and delete unused entities without downstream dependents.
Inspection	InspectList , InspectGet	Page through bounded document, entity, geometry, topology, and assembly collections, then retrieve canonical detail for one stable reference.
Entity	EntitySetPlacement	Apply rigid transforms in world or parent space to placeable CAD entities and component instances.
Sketch	SketchCreate , SketchAddLines , SketchAddCircle , SketchAddArc , SketchAddPoint , SketchConstrain , SketchProject , SketchPattern , SketchTrim	Create plane-bound sketches; add local lines, circles, arcs, and points; apply geometric and dimensional constraints; project external geometry; create linear or circular patterns; and trim at intersections.
Solid	SolidCreateBody , SolidExtrude , SolidRevolve , SolidSweep , SolidLoft , SolidBoolean , SolidRepeat , SolidFillet , SolidChamfer , SolidShell	Create feature bodies; add or cut material by extrusion, revolution, sweep, or loft; perform body booleans; repeat or mirror features and bodies; and apply fillets, chamfers, and inward or outward shells.
Assembly	AssemblyCreate , AssemblyAddComponent , AssemblySetGrounded , AssemblyMateAdd , AssemblyMateRemove , AssemblySolve	Create assemblies; insert placed component instances; ground or release components; add and remove geometry-referenced mates; and solve the resulting assembly constraints.

Appendix

Table 8: Canonical CAD operations implemented by the current FreeCAD backend. The 34 tools constitute the complete SDK backend registry used by DeciCAD .

Capability	FreeCAD	SolidWorks	AutoCAD
Documents and objects	✓	✓	✓
Inspection and query	✓	✓	✓
Parametric modeling	✓	✓
Assemblies and mates	✓	✓
2D drafting			✓
Exchange formats	STEP, STL	STEP, STL	DXF

Appendix

Table 9: Capabilities supported by the three CAD backends. All backends implement the same SDK contract; all experiments in this paper use the FreeCAD backend.

LLM Agent Harnesses

LLM agent harnesses are the software and runtime systems that organize an agent’s tools, context, state, execution, and safeguards.

See all 389 papers

TOOLCAD: Exploring Tool-Using Large Language Models in Text-to-CAD Generation with Reinforcement Learning
Apr 9, 2026Yifei Gong, Xing Wu, Wenda Liu +1Text-to-CAD Generation LLM Tool Use
School of Computer Engineering & Science, Shanghai University
RA-CAD: Learning Post-Execution Critique for State-Aware Text-to-CAD Generation
Aug 6, 2026Shuhao Yan, Changhao He, Peng Hu +1RL for Code Generation Code Generation
College of Computer Science, Sichuan University, Chengdu, China · School of Artificial Intelligence, Sichuan University, Chengdu, China · National Key Laboratory of Fundamental Algorithms and Models for Engineering Numerical Simulation, Sichuan University, Chengdu, China
Memory-Augmented Reinforcement Learning Agent for CAD Generation
May 19, 2026Yin Xiaolong, Liu Yu, Shen Jiahang +4Text-to-CAD Generation RL for Generative Models
Zhejiang University · Ocean University of China

Apr 9, 2026J/K move · Enter open · S save

TOOLCAD: Exploring Tool-Using Large Language Models in Text-to-CAD Generation with Reinforcement Learning

Yifei Gong, Xing Wu, Wenda Liu, Kang Tu

School of Computer Engineering & Science, Shanghai University

Text-to-CAD Generation LLM Tool Use LLM Agent Training RL for Tool Use

Open paper AskPDF arXiv

Computer-Aided Design (CAD) is an expert-level task that relies on long-horizon reasoning and coherent modeling actions. Large Language Models (LLMs) have shown remarkable advancements in enabling language agents to tackle real-world tasks. Notably, there has been no investigation into how tool-using LLMs optimally interact with CAD engines, hindering the emergence of LLM-based agentic text-to-CAD modeling systems. We propose ToolCAD, a novel agentic CAD framework deploying LLMs as tool-using agents for text-to-CAD generation. Furthermore, we introduce an interactive CAD modeling gym to rollout reasoning and tool-augmented interaction trajectories with the CAD engine, incorporating hybrid feedback and human supervision. Meanwhile, an end-to-end post-training strategy is presented to enable the LLM agent to elicit refined CAD Modeling Chain of Thought (CAD-CoT) and evolve into proficient CAD tool-using agents via online curriculum reinforcement learning. Our findings demonstrate ToolCAD fills the gap in adopting and training open-source LLMs for CAD tool-using agents, enabling them to perform comparably to proprietary models, paving the way for more accessible and robust autonomous text-to-CAD modeling systems.

Abstract

LLM Agent Harnesses

Explore similar work

TOOLCAD: Exploring Tool-Using Large Language Models in Text-to-CAD Generation with Reinforcement Learning

RA-CAD: Learning Post-Execution Critique for State-Aware Text-to-CAD Generation

Memory-Augmented Reinforcement Learning Agent for CAD Generation