Sketching in 3D space enables expressive reasoning about shape, structure, and spatial relationships, yet generating 3D sketches through natural language remains a major challenge. In this work, we introduce 3DrawAgent, a training-free, language-driven framework for 3D sketch generation that leverages large language models (LLMs) to sequentially draw 3D Bezier curves under geometric feedback. Unlike prior 2D sketch agents, our method introduces a relative experience optimization strategy that adapts the recently proposed Group Reward Policy Optimization (GRPO) paradigm. Instead of relying on explicit ground-truth supervision, we construct pairwise comparisons among generated sketches, with each pair consisting of a relatively better and a worse result based on CLIP-based perceptual rewards and LLM-based fine-grained qualitative assessment. These experiences are then used to iteratively refine the prior knowledge of 3D drawing, enabling black-box reinforcement of the model's 3D awareness. This design allows our model to self-improve its spatial understanding and drawing quality without parameter updates. Experiments show that 3DrawAgent can generate complex and coherent 3D Bezier sketches from diverse textual prompts, exhibit emergent geometric reasoning, and generalize to novel shapes, establishing a new paradigm for advancing the field of training-free 3D sketch intelligence.
Figures & tables
Figure 1: Top : Prior works typically rely on pre-trained diffusion models as 3D priors. Bottom: Our work performs training-free 3D sketch generation by refining an LLM’s spatial reasoning.
Figure 2: Framework Overview. Given a text prompt, our framework uses an LLM to autoregressively generate 3D Bezier curves. Each generated sketch is evaluated with a CLIP-based model to produce quality scores, forming contrastive pairs that teach the LLM which sketches are better or worse. These insights are accumulated into an experience library, which is then leveraged to guide subsequent language-driven 3D sketch generation, enabling coherent, semantically aligned, and spatially consistent 3D drawings.
Method
Train
Text-to-3D (Category)
Text-to-3D (Fine-Grained)
Image-to-3D
CLIP-S T
AES
CLIP-S T
AES
CLIP-S I
AES
Diff3DS [ 36 ]
✓
0.648
3.791
0.650
3.770
0.865
3.828
3Doodle [ 4 ]
✓
-
-
-
-
0.869
4.264
Dream3DVG [ 17 ]
✓
0.660
4.150
0.670
4.174
-
-
3DrawAgent ( DeepSeek-V3.2 )
✗
0.643
4.108
0.664
4.146
-
-
3DrawAgent ( Gemini-2.5 Pro )
✗
0.649
4.161
0.669
4.175
0.873
4.255
Table 1: Comparison results on Text-to-3D (category- and fine-grained) and Image-to-3D generation. “-”: not reported.
Table 2: Ablation study on the core components of our CKE pipeline. We report CLIP-S on the ModelNet40 test set. The Base model (Epoch 0) has no experience. Our method achieves strong results even without ground truth and benefits from a group size of K=5 .
Figure 4: Statistics analysis of 200 rollouts for a single 3D drawing task during contrastive knowledge extraction, uniformly sampled to 100 for visualization. (a) Average pairwise similarity between curves within each rollout. (b) Distribution of curve counts across the 200 rollouts. (c) Reward score distribution over the 200 rollouts. (d) Bracket-matching rate, computed as matched cases divided by total cases.
Figure 5: Extracted Knowledge Analysis.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Impact of Stroke Constraints on 3D Abstraction across Categories. We evaluate the model’s generation capability under varying Bezier curve budgets (rows from 8 to 128) across diverse categories: Bench , Chair , Plant , and Person . At minimal budgets (8 curves), the model performs high-level semantic abstraction, producing skeletal representations (e.g., a stick figure for the person or a simple stem for the plant). As the budget increases to 32–64 curves, structural details emerge, such as the pot geometry for the plant or parallel slats for the furniture. At 128 curves, the sketches evolve into dense wireframes. This demonstrates the model’s versatility in adapting its planning strategy from abstract symbolism to geometric fidelity for both rigid and organic shapes.
Method
GPU / API
min / Sample
Avg. Cost (USD)
3Doodle
NVIDIA RTX3090
∼ 120
0.80
Diff3DS
NVIDIA A10
∼ 120
1.50
Dream3DVG
NVIDIA A100
∼ 60
1.30
Ours
DeepSeek-V3.2
∼ 2
0.09
Appendix
Table 3: Cost comparison with single objects.
Metric
Unit Price (USD)
Total Volume
Est. Cost (USD)
Input (Cache Hit)
0.027 / 1M tokens
∼ 150M tokens
4.1
Input (Cache Miss)
0.275 / 1M tokens
∼ 10M tokens
2.8
Output
0.413 / 1M tokens
∼ 10M tokens
4.1
Total
-
-
11.0
Appendix
Table 4: Training-Free Cost Analysis (DeepSeek-V3.2-Exp). Costs are estimated for a complete experience extraction run (100 prompts, 3 epochs, K=5 ).
Setting
Component
Ep 0
Ep 1
Ep 2
Ep 3
Base
(w/o CKE)
0.5735
-
-
-
Random
(w/ CKE)
0.5735
0.6420
0.5595
0.6094
Ours
(w/ CKE)
0.5735
0.6461
0.6643
0.6428
Appendix
Table 5: Comparison of CKE against Random Selection. We report the CLIP-S T scores across different experience extraction epochs.
Figure 7: User Study Results. Percentage of user preference votes. 3DrawAgent (46.66%) is the most preferred method, showing a clear advantage over Dream3DVG (36.67%) and Diff3DS (16.67%) in terms of combined semantic and geometric quality.
Figure 8: Visual Examples of Common Failure Modes. Despite our experience bank’s guidance, 3DrawAgent encounters challenges with strict geometric connectivity and handling semantic ambiguity in complex structures. Key issues include (a) disconnected junctions where strokes should intersect, (b) floating components, and (c) visual clutter when managing ambiguous topological constraints.
Figure 9: Full System Prompt. Raw text input provided to the LLM, combining role definition, strict syntax constraints (code-only output), coordinate system rules, and a few-shot example (“A benz car”) to guide 3D sketch generation.
Figure 10: Data Flow in Experience Extraction. Raw logs collected during exploration, capturing the Agent’s outputs (3D curves) and the Environment’s feedback (rewards). These paired samples serve as the input to the “Contrastive Knowledge Extraction” module, where the “Judge LLM” derives spatial principles by contrasting high- and low-reward trajectories.