DeciCAD: Harness-Grounded Learning of Action Policies for CAD Agents
Organizations: University of Science and Technology of China · Nanyang Technological University
Abstract
Large Language Model agents operate through closed-loop action policies whose decision space is defined by an agent harness. We investigate how this operating space shapes agent behavior and policy learning in computer-aided design (CAD), a professional domain characterized by persistent state, geometric constraints, and long-horizon procedures. We instantiate this perspective in DeciCAD, a domain-native harness that exposes semantic CAD actions, decision-oriented observations, environment feedback, and persistent state transitions. Building on this harness, we develop a learning pipeline that extracts causal decision supervision from interaction trajectories and improves the policy through supervised fine-tuning (SFT) and geometry-grounded online reinforcement learning (RL). Across four CAD benchmarks, DeciCAD achieves the best F1 and intersection over union (IoU) while reducing model-output tokens by 45--88% relative to a general-purpose coding harness. Under matched supervision, its SFT policy improves F1 from 0.6883 to 0.7437 and outperforms the coding policy; subsequent online RL further raises geometry availability from 66.5% to 73.5% and F1 to 0.7880. These results demonstrate that the decision space induced by a harness materially affects both agent execution and policy learning in CAD.
Figures & tables
| CADPrompt | P3D-Text-to-3D | CADInstruct | P3D-Assembly-3D | |||||||||||||
| Method | CD | F1 | IoU | Avail. | CD | F1 | IoU | Avail. | CD | F1 | IoU | Avail. | CD | F1 | IoU | Avail. |
| Zero-shot | 0.32 | 61.84 | 74.10 | 93.50 | 0.35 | 37.11 | 48.15 | 90.00 | 0.93 | 19.28 | 23.24 | 100.00 | 0.52 | 25.40 | 22.35 | 71.92 |
| CADSmith | 0.26 | 63.70 | 72.08 | 100.00 | 0.33 | 40.74 | 52.73 | 100.00 | 0.85 | 20.11 | 23.14 | 100.00 | 0.64 | 26.27 | 21.19 | 96.06 |
| CAD-Assistant | 0.41 | 56.15 | 65.77 | 100.00 | 0.45 | 35.83 | 53.04 | 100.00 | 0.98 | 18.07 | 14.17 | 100.00 | 0.64 | 25.43 | 21.12 | 99.51 |
| Claude Code | 0.29 | 65.44 | 73.86 | 100.00 | 0.34 | 40.44 | 51.20 | 100.00 | 0.92 | 20.10 | 22.62 | 100.00 | 0.32 | 31.95 | 23.22 | 100.00 |
| DeciCAD | 0.28 | 65.83 | 75.12 | 100.00 | 0.25 | 43.63 | 54.22 | 100.00 | 0.84 | 23.49 | 23.35 | 100.00 | 0.49 | 32.11 | 27.07 | 100.00 |
| Metric | SDO | SDO-CFF | RSO | EFO |
|---|---|---|---|---|
| Avail. (%) | 100.00 | 95.00 | 55.00 | 87.00 |
| CD | 0.25 | 0.44 | 0.99 | 0.61 |
| F1 | 43.63 | 32.26 | 21.62 | 31.75 |
| IoU | 54.22 | 43.78 | 33.06 | 40.71 |
| Input tokens | 390.54 | 3,050.00 | 3,450.00 | 3,200.00 |
| Harness | Policy | Avail. (%) | F1 | IoU | CD |
|---|---|---|---|---|---|
| Coding | Base | 12.5 | 0.5902 | 0.5871 | 0.00972 |
| SFT | 60.5 | 0.6460 | 0.6286 | 0.01130 | |
| DeciCAD | Base | 61.5 | 0.6883 | 0.6487 | 0.01023 |
| SFT | 66.5 | 0.7437 | 0.6631 | 0.00782 |
| Harness | Policy | Avail. (%) | F1 | IoU | CD |
|---|---|---|---|---|---|
| Coding | SFT | 60.5 | 0.6460 | 0.6286 | 0.01130 |
| SFT+GRPO | 65.5 | 0.6253 | 0.5987 | 0.02253 | |
| DeciCAD | SFT | 66.5 | 0.7437 | 0.6631 | 0.00782 |
| SFT+GRPO | 73.5 | 0.7880 | 0.7036 | 0.00570 |
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
| Benchmark | Samples | Input | Task type |
|---|---|---|---|
| CADPrompt | 200 | Text | Part modeling |
| P3D-Text-to-3D | 100 | Text | Part modeling |
| CADInstruct | 100 | Text | Part modeling |
| P3D-Assembly-3D | 203 | Text + image | Assembly modeling |
| Component | Setting |
|---|---|
| Student / teacher | Qwen3.5-9B / Claude Sonnet 4.6 |
| SFT LoRA | rank 32, alpha 64, dropout 0.05 |
| SFT learning rate | , cosine |
| SFT context | 65,536 tokens |
| Checkpoint interval | 25 optimizer steps |
| GRPO LoRA | rank 32, alpha 64 |
| Harness | Audited tasks | Train / dev. | Target cap | Recovery targets |
|---|---|---|---|---|
| Coding | 499 | 471 / 28 | 12 | Up to 4 observed |
| DeciCAD | 642 | 542 / 100 | 11 + 3 | Up to 3 observed or controlled |
| Domain | Canonical operations | Supported behavior |
|---|---|---|
| Document | DocumentCreate , DocumentOpen , DocumentSave , DocumentExport , DocumentClose , DocumentDeleteEntity | Create and open native FreeCAD documents; save editable documents; export selected targets to STEP or STL; close documents; and delete unused entities without downstream dependents. |
| Inspection | InspectList , InspectGet | Page through bounded document, entity, geometry, topology, and assembly collections, then retrieve canonical detail for one stable reference. |
| Entity | EntitySetPlacement | Apply rigid transforms in world or parent space to placeable CAD entities and component instances. |
| Sketch | SketchCreate , SketchAddLines , SketchAddCircle , SketchAddArc , SketchAddPoint , SketchConstrain , SketchProject , SketchPattern , SketchTrim | Create plane-bound sketches; add local lines, circles, arcs, and points; apply geometric and dimensional constraints; project external geometry; create linear or circular patterns; and trim at intersections. |
| Solid | SolidCreateBody , SolidExtrude , SolidRevolve , SolidSweep , SolidLoft , SolidBoolean , SolidRepeat , SolidFillet , SolidChamfer , SolidShell | Create feature bodies; add or cut material by extrusion, revolution, sweep, or loft; perform body booleans; repeat or mirror features and bodies; and apply fillets, chamfers, and inward or outward shells. |
| Assembly | AssemblyCreate , AssemblyAddComponent , AssemblySetGrounded , AssemblyMateAdd , AssemblyMateRemove , AssemblySolve | Create assemblies; insert placed component instances; ground or release components; add and remove geometry-referenced mates; and solve the resulting assembly constraints. |
| Capability | FreeCAD | SolidWorks | AutoCAD |
|---|---|---|---|
| Documents and objects | ✓ | ✓ | ✓ |
| Inspection and query | ✓ | ✓ | ✓ |
| Parametric modeling | ✓ | ✓ | |
| Assemblies and mates | ✓ | ✓ | |
| 2D drafting | ✓ | ||
| Exchange formats | STEP, STL | STEP, STL | DXF |
LLM Agent Harnesses
LLM agent harnesses are the software and runtime systems that organize an agent’s tools, context, state, execution, and safeguards.
See all 389 papers